Morten's Dev

Emacs and the Language Server Protocol

2019-03-28T21:56:01+02:00

Having used Emacs since 2006, I never found a good, reliable, and general enough solution to code completion, symbol cross referencing, go-to definition until I stumbled upon the Language Server Protocol (LSP).

From the specification:

The Language Server protocol is used between a tool (the client) and a language smartness provider (the server) to integrate features like auto complete, go to definition, find all references and alike into the tool.

It tackles the code intelligence problem quite nicely: To support a given programming language with LSP, a language server program must be implemented. When running, all editors and IDEs supporting LSP will be able to employ its features. Adoption is already very good across the field. ¹

LSP reduces an $m \times n$ complexity problem to $m + n$: the latter refers to only needing a language server for each language and a client for each IDE, whereas the former means that for every language each IDE would require, in most cases, standalone support.

1. Prerequisites

In this article, I will go over the configuration and setup of Emacs for LSP support of C++, Python, and Rust. It is assumed that Emacs version 25.1 or later is already installed and setup for basic usage, and that you’re familiar with editing its config file. Additionally, I use macOS but the installation commands can easily be translated to other solutions and other OSs.

2. Emacs configuration

First install the lsp-mode package, which is the LSP client that we will be using:

M-x package-install ↵ lsp-mode ↵

The initial configuration is very simple: ²

(use-package lsp-mode  :config  (add-hook 'c++-mode-hook #'lsp)  (add-hook 'python-mode-hook #'lsp)  (add-hook 'rust-mode-hook #'lsp))

lsp-mode comes prepackaged to use certain available LSP servers, that it will automatically spin up when a mode requests it (by invoking lsp). If LSP servers were already installed at this point, then you would be all set.

2.1. C++

Clangd is currently the best working language server for C++ for the projects I work on. Even though cquery comes as a default in lsp-mode, and has more features than clangd, there were times that it would fail to find known symbols for large projects, and both the index generation time and size were longer and bigger than clangd’s.

The first working version of clangd is LLVM v7 but I’ve had the best results with v9 or later (at the time of writing v10 is default in Homebrew), so install LLVM and thus clangd:

% brew install llvm % which clangd /usr/local/opt/llvm/bin/clangd

Edit 2020-04-21: No longer using --HEAD since v10 is default.

Add the following to the lsp-mode config:

(use-package lsp-mode  :config  ;; `-background-index' requires clangd v8+!  (setq lsp-clients-clangd-args '("-j=4" "-background-index" "-log=error"))  ;; ..  )

It tells clangd that it can use 4 concurrent jobs and to make a complete background index on disk. Without -background-index, it will only keep an in-memory index of the files that are active in Emacs buffers, but to be able to find references and symbols in any project file the background index is recommended. The background index, named “.clangd” by default, is placed at the root of the project.

Clangd tries to locate the “compile_commands.json” file in the root of the project, but if you normally create separate build folders it is useful to place a symlink pointing to where “compile_commands.json” is located inside it. For instance, if the build folder is “build/debug/” the following symlink could be created at the project root:

compile_commands.json -> build/debug/compile_commands.json

Most build tools can output “compile_commands.json”. In CMake you write:

set(CMAKE_EXPORT_COMPILE_COMMANDS ON)

Edit 2020-04-21: Improved text referring to the background index and the symlinking of “compile_commands.json”.

2.2. Python

For Python, the default language server is pyls:

% pip install python-language-server % which pyls /usr/local/bin/pyls

That’s it.

2.3. Rust

The default language server for Rust is rls which is installed via rustup. Before installing rustup blindly, first download the shell script and look it over, then proceed:

% curl https://sh.rustup.rs -sSf | sh % rustup update % rustup component add rls rust-analysis rust-src % which rls ~/.cargo/bin/rls

Skip the rustup installation if Rust is already installed.

3. Usage

Using LSP is easy: open a source code file with one of the major modes associated the lsp hook. If no running language server is handling the particular project, a new one will be spun up to communicate with lsp-mode.

When LSP does not know which project to associate a file, usually the first time visiting a file, it will ask what to do. In this case I’m testing using my vermin project:

vermin.py is not part of any project. Select action:

The following actions are possible:

Do not ask more for the current project (add “~/git/vermin/” to lsp-session-folder-blacklist)
Do not ask more for the current project (select ignore path interactively).
Do nothing and ask me again when opening other files from the folder.
Import project by selecting root directory interactively.
Import project root ~/git/vermin/

Usually the last option is what you want. Simply write Import ↹ root ↹ ↵.

The minibuffer will display (the PID will differ):

LSP :: Connected to [pyls:94242 status:starting].

To try out some LSP functionality, take a look at the vermin’s entry point:

#!/usr/bin/env python from vermin.main import main if __name__ == "__main__": main()

Put the cursor at “main” right after import on the second line. We want to find the definition of that function: M-x lsp-find-definition ↵. Voila, that jumps to line 12 of “vermin/main.py”:

# .. def main(): # Cursor will be at 'm'. config = Config.get() args = parse_args(sys.argv[1:]) # ..

The minibuffer will show that it connected to the same pyls instance:

LSP :: Connected to [pyls:94242].

Next, let’s try to find references of parse_args() by placing the cursor on it and
M-x lsp-find-references ↵.

The results are displayed this way due to the helm and helm-xref packages.

Other useful commands:

lsp-describe-thing-at-point: displays the full documentation of the thing at point
lsp-rename: renames the symbol (and all references to it) under point to a new name
lsp-execute-code-action: executes code action, like “did you mean X instead of Y?”
lsp-describe-session: describes current LSP session and its capabilities

Take a look here for more information about extensions and features.

Edit 2019-04-01: My .emacs.d is freely available for the complete LSP configuration, and lots of other stuff related to Emacs.

4. Extras

This section is dedicated to extra features and niceties that I find makes LSP more usable and powerful.

4.1. `lsp-ui`

lsp-ui adds higher level UI modules, like documentation child frame while hovering a symbol, integration with flycheck, and enables inline definition and references peeking - just to name a few things.

My configuration relies on flycheck being set up:

(use-package lsp-mode  ;; ..  :config  (setq lsp-prefer-flymake nil) ;; Prefer using lsp-ui (flycheck) over flymake.  ;; ..  ) (use-package lsp-ui  :requires lsp-mode flycheck  :config  (setq lsp-ui-doc-enable t  lsp-ui-doc-use-childframe t  lsp-ui-doc-position 'top  lsp-ui-doc-include-signature t  lsp-ui-sideline-enable nil  lsp-ui-flycheck-enable t  lsp-ui-flycheck-list-position 'right  lsp-ui-flycheck-live-reporting t  lsp-ui-peek-enable t  lsp-ui-peek-list-width 60  lsp-ui-peek-peek-height 25)  (add-hook 'lsp-mode-hook 'lsp-ui-mode))

4.2. `hydra`

Another thing that’s really useful, not just related to LSP, is the hydra package, which is described very well by its author:

The Hydra is vanquished once Hercules, any binding that isn’t the Hydra’s head, arrives. Note that Hercules, besides vanquishing the Hydra, will still serve his original purpose, calling his proper command. This makes the Hydra very seamless, it’s like a minor mode that disables itself auto-magically.

In essence, it yields a menu with keybindings in the minibuffer, a hydra, which won’t be shown if the bindings are pressed fast enough.

The following configuration is trimmed down to only show parts related to LSP:

(use-package hydra) (use-package helm) (use-package helm-lsp  :config  (defun netrom/helm-lsp-workspace-symbol-at-point ()  (interactive)  (let ((current-prefix-arg t))  (call-interactively #'helm-lsp-workspace-symbol)))  (defun netrom/helm-lsp-global-workspace-symbol-at-point ()  (interactive)  (let ((current-prefix-arg t))  (call-interactively #'helm-lsp-global-workspace-symbol)))) (use-package lsp-mode  :requires hydra helm helm-lsp  ;; ..  (setq netrom--general-lsp-hydra-heads  '(;; Xref  ("d" xref-find-definitions "Definitions" :column "Xref")  ("D" xref-find-definitions-other-window "-> other win")  ("r" xref-find-references "References")  ("s" netrom/helm-lsp-workspace-symbol-at-point "Helm search")  ("S" netrom/helm-lsp-global-workspace-symbol-at-point "Helm global search")  ;; Peek  ("C-d" lsp-ui-peek-find-definitions "Definitions" :column "Peek")  ("C-r" lsp-ui-peek-find-references "References")  ("C-i" lsp-ui-peek-find-implementation "Implementation")  ;; LSP  ("p" lsp-describe-thing-at-point "Describe at point" :column "LSP")  ("C-a" lsp-execute-code-action "Execute code action")  ("R" lsp-rename "Rename")  ("t" lsp-goto-type-definition "Type definition")  ("i" lsp-goto-implementation "Implementation")  ("f" helm-imenu "Filter funcs/classes (Helm)")  ("C-c" lsp-describe-session "Describe session")  ;; Flycheck  ("l" lsp-ui-flycheck-list "List errs/warns/notes" :column "Flycheck"))  netrom--misc-lsp-hydra-heads  '(;; Misc  ("q" nil "Cancel" :column "Misc")  ("b" pop-tag-mark "Back")))  ;; Create general hydra.  (eval `(defhydra netrom/lsp-hydra (:color blue :hint nil)  ,@(append  netrom--general-lsp-hydra-heads  netrom--misc-lsp-hydra-heads)))  (add-hook 'lsp-mode-hook  (lambda () (local-set-key (kbd "C-c C-l") 'netrom/lsp-hydra/body))))

To activate the hydra, press C-c C-l with the cursor placed somewhere relevant:

Edit 2019-04-01: Using helm-imenu instead of lsp-ui-imenu because the former enables easy filtering via Helm.

4.3. `company-lsp`

For completions at cursor, I use company-mode with company-lsp for the purposes of LSP:

(use-package company  :config  (setq company-idle-delay 0.3)  (global-company-mode 1)  (global-set-key (kbd "C-<tab>") 'company-complete)) (use-package company-lsp  :requires company  :config  (push 'company-lsp company-backends)  ;; Disable client-side cache because the LSP server does a better job.  (setq company-transformers nil  company-lsp-async t  company-lsp-cache-candidates nil))

See lists of server implementations here and here, and client implementations. ↩︎
use-package is great for automatically downloading and installing packages. ↩︎

Sigs: Migrating to C++17

2019-03-21T23:19:20+02:00

sigs has been migrated to using C++17 from C++14. Didn’t take much doing other than some CMake configuration tweakings, and to update the Google Test framework to version 1.8.1.

1. CMake configuration changes

Setting the required C++ standard to C++17 was done with:

set(CMAKE_CXX_STANDARD 17) set(CMAKE_CXX_STANDARD_REQUIRED ON)

And was tested to work with Clang 6 + 7, GCC 7 + 8, and VS 2017.

2. C++17 features

The fun part was finding new features to put into action in the project.

2.1. [[nodiscard]]

In certain cases it is important that the returned value is not discarded, which is exactly what [[nodiscard]] achieves. For example, in the following a warning will be shown if the return value of Signal::interface() isn’t used:

[[nodiscard]] inline std::unique_ptr<Interface> interface() noexcept {  return std::make_unique<Interface>(this); }

2.2. Initializer statements

Having the means to include initialization in if and switch statements is a very nice, new feature that helps scoping variables for intent. If an initialization is made before the statement then it will be live in the parent scope of that statement, a scope that might span for more than what was intended.

There were no places in sigs where a scope of an initializer to an if or switch statement spanned more than was necessary, but still, there were four places to put it into effect. An example:

if (auto *sig = entry.signal(); sig) {  (*sig)(retFunc, std::forward<Args>(args)...); }

2.3. std::optional

The concept of an optional value is a nice abstraction that enables one to not use any value, from the return value space, to act as an indicator of null or an error. For instance, if the return type of a function is int then an error might be signaled by using -1, but that also means that the value cannot be used as a valid value. Instead, std::optional<T> can be seen as a bool for indicating an error and the encapsulated value T if defined.

An example usage from the project:

template <typename Ret, typename... Args> class Signal<Ret(Args...)> final { public:  // ..  void disconnect(std::optional<Connection> conn) noexcept  {  // ..  } // ..

This solution is clearer and better than the previous one:

void disconnect(Connection conn = nullptr)

2.4. std::is_void_v

std::is_void was introduced already in C++11 and std::is_void_v is basically just a helper variable template:

template <typename T> inline constexpr bool is_void_v = is_void<T>::value;

In sigs, I ended up replacing the local isVoidReturn() with direct usage due to the shortness of the statement:

static_assert(!std::is_void_v<ReturnType>, "Must have non-void return type!");

2.5. noexcept

Every function in C++ is either potentially throwing or non-throwing. noexcept is interesting because it marks a function as non-throwing, and informs that exceptions aren’t being used intentionally. If an exception is thrown anyway, std::terminate is invoked.

noexcept was added in C++11 but I didn’t put it to use until now. It made it possible for the compiler to optimize further and reduced the size of the binaries somewhat. Can’t say no to that. I found 34 places to add it, like:

Signal() noexcept = default;

Sigs: Return Values

2019-03-08T21:36:12+02:00

sigs, my simple thread-safe signal/slot C++ library, has been extended to support slot functions with non-void return types. Such return values can be gathered by triggering the signal with a function. But the argument type must be the same as the return type!

The following example adds together the integers from each connected slot:

sigs::Signal<int()> s; s.connect([] { return 1; }); s.connect([] { return 2; }); s.connect([] { return 3; }); int sum = 0; s([&sum](int retVal) { sum += retVal; }); // sum is now = 1 + 2 + 3 = 6

This enables a way for each slot to contribute to a shared computation.

In order to implement this, the Signal::operator(..) must be able to assert that the provided function either has no argument, representing a slot with no return type, or a non-void argument for a slot with one. Enter VoidableFunction:

template <typename T> class VoidableFunction { public:  using func = std::function<void(T)>; }; /// Specialization for void return types. template <> class VoidableFunction<void> { public:  using func = std::function<void()>; }; template <typename Ret, typename... Args> class Signal<Ret(Args...)> {  using Slot = std::function<Ret(Args...)>;  // .. public:  using ReturnType = Ret;  // ..  template <typename RetFunc = typename VoidableFunction<ReturnType>::func>  void operator()(const RetFunc &retFunc, Args &&... args)  {  // ..

A specialization was necessary for void return types due to the lack of a value: a value is only be provided in the non-void case, which is what typename RetFunc = typename VoidableFunction<ReturnType>::func yields.

The isVoidReturn() is used to statically assert at compile-time that no void return type is used:

 template <typename RetFunc = typename VoidableFunction<ReturnType>::func>  void operator()(const RetFunc &retFunc, Args &&... args)  {  static_assert(!isVoidReturn(), "Must have non-void return type!");  // ..  } private:  static constexpr bool isVoidReturn()  {  return std::is_void<ReturnType>::value;  }

This is necessary since the prototype must match both void and non-void types such that it can yield a sensible error in the former case.

For each connected slot, the return value function is called with the result of the slot function, or passed to chained signals:

Lock lock(entriesMutex); for (auto &entry : entries) {  auto *sig = entry.signal();  if (sig) {  (*sig)(retFunc, std::forward<Args>(args)...);  }  else {  retFunc(entry.slot()(std::forward<Args>(args)...));  } }

Revamped and new Domain

2019-03-03T13:30:08+02:00

I moved my blog to the new domain mortens.dev using Pelican as the static page generator. The blog is still a collection of articles and ramblings about programming, security, algorithms, and technology in general.

Patching Binaries: Live Modification

2016-09-07T23:35:59+02:00

Patching Binaries is a series of articles about how to extract information and modify program behavior. It focuses on the Mac Mach-O executable format for the x86-64 architecture, but the techniques are similar for other formats.

Before proceeding it is recommended reading the second part about jumping and bypassing instructions.

The files used in this article can be found here.

This article is about how to change data values of a program and how to read from and write to memory of a live program to change its behavior.

Compile “score.cc” as “score” and strip it. Let’s see what it does:

% ./score score = 1

Think of a game that displays your high score. We want to change it so use “lldb_entrypoint.sh” to start it up:

% ./lldb_entrypoint.sh score (lldb) target create "score" Current executable set to 'score' (x86_64). (lldb) command source -s 0 '/tmp/.lldbcmds' Executing commands in '/tmp/.lldbcmds'. (lldb) b 0x0000000100001140 Breakpoint 1: address = 0x0000000100001140 (lldb) r Process 38693 launched: '/Users/netrom/git/patching/live/score' (x86_64) Process 38693 stopped * thread #1: tid = 0x28e293, 0x0000000100001140 score`___lldb_unnamed_function1$$score, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1  frame #0: 0x0000000100001140 score`___lldb_unnamed_function1$$score score`___lldb_unnamed_function1$$score: -> 0x100001140 <+0>: pushq %rbp  0x100001141 <+1>: movq %rsp, %rbp  0x100001144 <+4>: subq $0x10, %rsp  0x100001148 <+8>: movq 0xeb1(%rip), %rdi ; (void *)0x00007fff78ba32f8: std::__1::cout (lldb) dis -b score`___lldb_unnamed_function1$$score: -> 0x100001140 <+0>: 55 pushq %rbp  0x100001141 <+1>: 48 89 e5 movq %rsp, %rbp  0x100001144 <+4>: 48 83 ec 10 subq $0x10, %rsp  0x100001148 <+8>: 48 8b 3d b1 0e 00 00 movq 0xeb1(%rip), %rdi ; (void *)0x00007fff78ba32f8: std::__1::cout  0x10000114f <+15>: 48 8d 35 26 0e 00 00 leaq 0xe26(%rip), %rsi ; "score = "  0x100001156 <+22>: c7 45 fc 00 00 00 00 movl $0x0, -0x4(%rbp)  0x10000115d <+29>: c7 45 f8 01 00 00 00 movl $0x1, -0x8(%rbp)  0x100001164 <+36>: e8 37 00 00 00 callq 0x1000011a0 ; ___lldb_unnamed_function2$$score  0x100001169 <+41>: 8b 75 f8 movl -0x8(%rbp), %esi  0x10000116c <+44>: 48 89 c7 movq %rax, %rdi  0x10000116f <+47>: e8 2c 0c 00 00 callq 0x100001da0 ; symbol stub for: std::__1::basic_ostream<char, std::__1::char_traits<char> >::operator<<(int)  0x100001174 <+52>: 48 8d 35 0a 0e 00 00 leaq 0xe0a(%rip), %rsi ; "'\n'"  0x10000117b <+59>: 48 89 c7 movq %rax, %rdi  0x10000117e <+62>: e8 1d 00 00 00 callq 0x1000011a0 ; ___lldb_unnamed_function2$$score  0x100001183 <+67>: 31 c9 xorl %ecx, %ecx  0x100001185 <+69>: 48 89 45 f0 movq %rax, -0x10(%rbp)  0x100001189 <+73>: 89 c8 movl %ecx, %eax  0x10000118b <+75>: 48 83 c4 10 addq $0x10, %rsp  0x10000118f <+79>: 5d popq %rbp  0x100001190 <+80>: c3 retq

The score of value 1 is passed to to -0x8(%rbp) at 0x10000115d. At offset 0x115d+3=0x1160 an 32-bit integer is passed as 4 bytes 1 0 0 0 (little endian).

Let’s say we want to change this value to 0x4030201 = 67305985. First the number must be split up into the 4 bytes: 1 2 3 4 (remember the reverse order here!). Then we can patch the program:

% ./patch_bytes.py score 0x1160 1 2 3 4 Patching "score" at offset 4448 with: [1, 2, 3, 4] (4 bytes) Read 14348 bytes Changing values at 4448 to 4452 Writing new data % ./score score = 67305985

Inspecting the instructions yields what we expect:

(lldb) dis -b -c 1 -s 0x10000115d score`___lldb_unnamed_function1$$score:  0x10000115d <+29>: c7 45 f8 01 02 03 04 movl $0x4030201, -0x8(%rbp) ; imm = 0x4030201

Recompile and strip “score.cc” to get back to square one.

We can also read and write registers of a program while it is running!

So let’s try changing the score right after it is set to 1 but before being used. As we saw before the value is saved into -0x8(%rbp) which we first have to understand what it means. The RBP (base pointer) register holds the pointer to the beginning of a stack frame while executing a function. Allocating on the stack is done by using a proper offset from the RBP, and since addresses grow upwards we use a negative addressing of -0x8 in this case.

In order to go further we need the value of the RBP register:

% lldb score (lldb) target create "score" Current executable set to 'score' (x86_64). (lldb) b 0x100001164 Breakpoint 1: address = 0x0000000100001164 (lldb) r Process 40054 launched: '/Users/netrom/git/patching/live/score' (x86_64) Process 40054 stopped * thread #1: tid = 0x298ab1, 0x0000000100001164 score`___lldb_unnamed_function1$$score + 36, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1  frame #0: 0x0000000100001164 score`___lldb_unnamed_function1$$score + 36 score`___lldb_unnamed_function1$$score: -> 0x100001164 <+36>: callq 0x1000011a0 ; ___lldb_unnamed_function2$$score  0x100001169 <+41>: movl -0x8(%rbp), %esi  0x10000116c <+44>: movq %rax, %rdi  0x10000116f <+47>: callq 0x100001da0 ; symbol stub for: std::__1::basic_ostream<char, std::__1::char_traits<char> >::operator<<(int) (lldb) reg read rbp  rbp = 0x00007fff5fbff5e0

Then we want to verify that -0x8(%rbp) has value 1:

(lldb) p *(int*)(0x00007fff5fbff5e0 - 0x8) (int) $0 = 1

That might look obscure but the command means to print the deferenced value of an integer pointer address at -0x8(%rbp).

Changing the value can be done in the following way:

(lldb) p *(int*)(0x00007fff5fbff5e0 - 0x8) = 42 (int) $1 = 42

Letting the program continue will show that it worked:

(lldb) c Process 40054 resuming score = 42 Process 40054 exited with status = 0 (0x00000000)

Note that the way to change the value could also have been written as:

(lldb) p *(int*)(`$rbp`-0x8) = 42

Which makes it possible to read the register without copying the address by hand!

How much easier would it be if we could automate this? A lot, I agree. So create a file called “setvalue.lldb” with the following contents:

b 0x100001164 r p *(int*)(`$rbp`-0x8) = 42 c exit

And run it like this:

% lldb -s setvalue.lldb score (lldb) target create "score" Current executable set to 'score' (x86_64). (lldb) command source -s 0 'setvalue.lldb' Executing commands in '/Users/netrom/git/patching/live/setvalue.lldb'. (lldb) b 0x100001164 Breakpoint 1: address = 0x0000000100001164 (lldb) r Process 40286 stopped * thread #1: tid = 0x29c96b, 0x0000000100001164 score`___lldb_unnamed_function1$$score + 36, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1  frame #0: 0x0000000100001164 score`___lldb_unnamed_function1$$score + 36 score`___lldb_unnamed_function1$$score: -> 0x100001164 <+36>: callq 0x1000011a0 ; ___lldb_unnamed_function2$$score  0x100001169 <+41>: movl -0x8(%rbp), %esi  0x10000116c <+44>: movq %rax, %rdi  0x10000116f <+47>: callq 0x100001da0 ; symbol stub for: std::__1::basic_ostream<char, std::__1::char_traits<char> >::operator<<(int) Process 40286 launched: '/Users/netrom/git/patching/live/score' (x86_64) (lldb) p *(int*)(`$rbp`-0x8) = 42 (int) $1 = 42 (lldb) c score = 42 Process 40286 resuming Process 40286 exited with status = 0 (0x00000000) (lldb) exit

Another approach is to start LLDB in a wait-for-process state where it will attach once a program with the given process name has been started. Something to note, though, is that the address 0x100001164 of before won’t work in this example due to how it attaches to the process. Instead we use the fact that main() is the first function so we can name it in LLDB as ___lldb_unnamed_function1$$score. Also, since it attaches to a running program we don’t have to start the process so we just continue after setting the break point. Finally, because we know that the original 0x100001164 is 36 instructions after the start of main() we set a break point using the RIP (instruction pointer) register with $rip+36.

Create file “waitsetvalue.lldb” with the following contents:

b score`___lldb_unnamed_function1$$score c b `$rip+36` c p *(int*)(`$rbp`-0x8) = 42 c exit

Next we fire up LLDB in one terminal (note it has to run as root/sudo to have privileges):

% sudo lldb -w -s waitsetvalue.lldb -n score (lldb) process attach --name "score" --waitfor

It is now in a waiting mode, so in the second terminal we simply start the program:

% ./score

LLDB will immediately attach in the other terminal:

(lldb) command source -s 0 'waitsetvalue.lldb' Executing commands in '/Users/netrom/git/patching/live/waitsetvalue.lldb'. (lldb) b score`___lldb_unnamed_function1$$score Breakpoint 1: where = score`___lldb_unnamed_function1$$score, address = 0x000000010ec6c140 (lldb) c Process 44506 resuming Process 44506 stopped * thread #1: tid = 0x2a93a3, 0x000000010ec6c140 score`___lldb_unnamed_function1$$score, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1  frame #0: 0x000000010ec6c140 score`___lldb_unnamed_function1$$score score`___lldb_unnamed_function1$$score: -> 0x10ec6c140 <+0>: pushq %rbp  0x10ec6c141 <+1>: movq %rsp, %rbp  0x10ec6c144 <+4>: subq $0x10, %rsp  0x10ec6c148 <+8>: movq 0xeb1(%rip), %rdi ; (void *)0x00007fff78ba32f8: std::__1::cout (lldb) b `$rip+36` Breakpoint 2: where = score`___lldb_unnamed_function1$$score + 36, address = 0x000000010ec6c164 (lldb) c Process 44506 resuming Process 44506 stopped * thread #1: tid = 0x2a93a3, 0x000000010ec6c164 score`___lldb_unnamed_function1$$score + 36, queue = 'com.apple.main-thread', stop reason = breakpoint 2.1  frame #0: 0x000000010ec6c164 score`___lldb_unnamed_function1$$score + 36 score`___lldb_unnamed_function1$$score: -> 0x10ec6c164 <+36>: callq 0x10ec6c1a0 ; ___lldb_unnamed_function2$$score  0x10ec6c169 <+41>: movl -0x8(%rbp), %esi  0x10ec6c16c <+44>: movq %rax, %rdi  0x10ec6c16f <+47>: callq 0x10ec6cda0 ; symbol stub for: std::__1::basic_ostream<char, std::__1::char_traits<char> >::operator<<(int) (lldb) p *(int*)(`$rbp`-0x8) = 42 (int) $2 = 42 (lldb) c Process 44506 resuming Process 44506 exited with status = 0 (0x00000000) (lldb) exit

Finally, in the second terminal, it will output:

score = 42

Phew!

The last example is attaching to a program after it has started. Compile “looper.cc” as “looper” and strip it. Then run it in a terminal:

% ./looper score = 20 score = 20 score = 20 ...

Open another terminal and attach to the process:

% lldb -p `pgrep looper` (lldb) process attach --pid 49530 Process 49530 stopped * thread #1: tid = 0x2b5010, 0x00007fff9218e10a libsystem_kernel.dylib`__semwait_signal + 10, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP  frame #0: 0x00007fff9218e10a libsystem_kernel.dylib`__semwait_signal + 10 libsystem_kernel.dylib`__semwait_signal: -> 0x7fff9218e10a <+10>: jae 0x7fff9218e114 ; <+20>  0x7fff9218e10c <+12>: movq %rax, %rdi  0x7fff9218e10f <+15>: jmp 0x7fff921887f2 ; cerror  0x7fff9218e114 <+20>: retq Executable module set to "/Users/netrom/git/patching/live/./looper". Architecture set to: x86_64h-apple-macosx. (lldb)

To get our bearings we backtrace, select the main looper function and disassemble:

(lldb) bt * thread #1: tid = 0x2b5010, 0x00007fff9218e10a libsystem_kernel.dylib`__semwait_signal + 10, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP  * frame #0: 0x00007fff9218e10a libsystem_kernel.dylib`__semwait_signal + 10  frame #1: 0x00007fff8d13ed0f libsystem_c.dylib`nanosleep + 199  frame #2: 0x00007fff8d13eb6f libsystem_c.dylib`sleep + 42  frame #3: 0x000000010aa3c171 looper`___lldb_unnamed_function1$$looper + 81  frame #4: 0x00007fff9a14a5ad libdyld.dylib`start + 1 (lldb) f 3 frame #3: 0x000000010aa3c171 looper`___lldb_unnamed_function1$$looper + 81 looper`___lldb_unnamed_function1$$looper:  0x10aa3c171 <+81>: movl %eax, -0x14(%rbp)  0x10aa3c174 <+84>: jmp 0x10aa3c136 ; <+22>  0x10aa3c179 <+89>: nop  0x10aa3c17a <+90>: nop (lldb) dis looper`___lldb_unnamed_function1$$looper:  0x10aa3c120 <+0>: pushq %rbp  0x10aa3c121 <+1>: movq %rsp, %rbp  0x10aa3c124 <+4>: subq $0x20, %rsp  0x10aa3c128 <+8>: movl $0x0, -0x4(%rbp)  0x10aa3c12f <+15>: movl $0x14, -0x8(%rbp)  0x10aa3c136 <+22>: movq 0xec3(%rip), %rdi ; (void *)0x00007fff78ba32f8: std::__1::cout  0x10aa3c13d <+29>: leaq 0xe2c(%rip), %rsi ; "score = "  0x10aa3c144 <+36>: callq 0x10aa3c180 ; ___lldb_unnamed_function2$$looper  0x10aa3c149 <+41>: movl -0x8(%rbp), %esi  0x10aa3c14c <+44>: movq %rax, %rdi  0x10aa3c14f <+47>: callq 0x10aa3cd80 ; symbol stub for: std::__1::basic_ostream<char, std::__1::char_traits<char> >::operator<<(int)  0x10aa3c154 <+52>: leaq 0xe1e(%rip), %rsi ; "'\n'"  0x10aa3c15b <+59>: movq %rax, %rdi  0x10aa3c15e <+62>: callq 0x10aa3c180 ; ___lldb_unnamed_function2$$looper  0x10aa3c163 <+67>: movl $0x1, %edi  0x10aa3c168 <+72>: movq %rax, -0x10(%rbp)  0x10aa3c16c <+76>: callq 0x10aa3cdb6 ; symbol stub for: sleep  0x10aa3c171 <+81>: movl %eax, -0x14(%rbp)  0x10aa3c174 <+84>: jmp 0x10aa3c136 ; <+22>

Since the program is already doing loops we have to break somewhere inside it, and it makes most sense to do it right where it reads the score value:

(lldb) dis -b -c 1 -s 0x10aa3c149 looper`___lldb_unnamed_function1$$looper:  0x10aa3c149 <+41>: 8b 75 f8 movl -0x8(%rbp), %esi

So we set a break point at 0x10aa3c149 and reach it:

(lldb) b 0x10aa3c149 Breakpoint 1: where = looper`___lldb_unnamed_function1$$looper + 41, address = 0x000000010aa3c149 (lldb) c Process 49530 resuming Process 49530 stopped * thread #1: tid = 0x2b5010, 0x000000010aa3c149 looper`___lldb_unnamed_function1$$looper + 41, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1  frame #0: 0x000000010aa3c149 looper`___lldb_unnamed_function1$$looper + 41 looper`___lldb_unnamed_function1$$looper: -> 0x10aa3c149 <+41>: movl -0x8(%rbp), %esi  0x10aa3c14c <+44>: movq %rax, %rdi  0x10aa3c14f <+47>: callq 0x10aa3cd80 ; symbol stub for: std::__1::basic_ostream<char, std::__1::char_traits<char> >::operator<<(int)  0x10aa3c154 <+52>: leaq 0xe1e(%rip), %rsi ; "'\n'"

The last thing to do is change the value and detach from the process:

(lldb) p *(int*)(`$rbp`-8) = 42 (int) $1 = 42 (lldb) detach Process 49530 detached

Finally, in the first terminal, it continually uses the changed value:

... score = 20 score = 42 score = 42 score = 42 ...

That concludes data manipulation of live processes.

Patching Binaries: Jumping The Fence

2016-09-06T21:43:55+02:00

Before proceeding it is recommended reading the first part about the fundamentals of finding and changing constant strings in binaries.

The files used in this article can be found here.

An approach to change the behavior of a program is to jump or bypass instructions. Or in other words: jumping the fence!

Compile and strip example program “verify.cc” as “verify” and run it:

% ./verify Serial number: 83 Incorrect! % ./verify Serial number: 90 Correct!

If we disregard knowing from the source code how the numbers are verified, we would like to somehow bypass the check so that all inputs are deemed correct. There are numerous ways of doing this but the easiest is putting NOP (No Operation - opcode 0x90) instructions at tactical places in the program. They are a means of masking out functionality because they simply do nothing.

First we need to find out where to place the NOPs. And as a preliminary deviation we will look at how to determine a program’s entry point address; where the first program instruction resides. When dealing with stripped executables we can’t break at main anymore.

In Mach-O binaries the program instructions are located in section __text of segment __TEXT:

% otool -s __TEXT __text verify | head -n 3 verify: Contents of (__TEXT,__text) section 00000001000010c0 55 48 89 e5 48 83 ec 30 48 8b 3d 39 0f 00 00 48

To easily retrieve the entry point and start LLDB with a break point on that address, take a look at “lldb_entrypoint.sh”:

#!/bin/sh ADDR=`otool -s __TEXT __text $1 | head -n 3 | tail -n 1 | awk '{print $1;}'` echo "b 0x${ADDR}" > /tmp/.lldbcmds lldb -s /tmp/.lldbcmds $1

With that let’s start up the debugger with our program and reach the break point:

% ./lldb_entrypoint.sh verify (lldb) target create "verify" Current executable set to 'verify' (x86_64). (lldb) command source -s 0 '/tmp/.lldbcmds' Executing commands in '/tmp/.lldbcmds'. (lldb) b 0x00000001000010c0 Breakpoint 1: address = 0x00000001000010c0 (lldb) r Process 88913 launched: '/Users/netrom/git/patching/jumps/verify' (x86_64) Process 88913 stopped * thread #1: tid = 0x233953, 0x00000001000010c0 verify`___lldb_unnamed_symbol1$$verify, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1  frame #0: 0x00000001000010c0 verify`___lldb_unnamed_symbol1$$verify verify`___lldb_unnamed_symbol1$$verify: -> 0x1000010c0 <+0>: pushq %rbp  0x1000010c1 <+1>: movq %rsp, %rbp  0x1000010c4 <+4>: subq $0x30, %rsp  0x1000010c8 <+8>: movq 0xf39(%rip), %rdi ; (void *)0x00007fff78ba32f8: std::__1::cout (lldb)

The next step is to disassemble to get an overview:

(lldb) dis verify`___lldb_unnamed_symbol1$$verify: -> 0x1000010c0 <+0>: pushq %rbp  0x1000010c1 <+1>: movq %rsp, %rbp  0x1000010c4 <+4>: subq $0x30, %rsp  0x1000010c8 <+8>: movq 0xf39(%rip), %rdi ; (void *)0x00007fff78ba32f8: std::__1::cout  0x1000010cf <+15>: leaq 0xe86(%rip), %rsi ; "Serial number: "  0x1000010d6 <+22>: movl $0x0, -0x4(%rbp)  0x1000010dd <+29>: callq 0x100001150 ; ___lldb_unnamed_symbol2$$verify  0x1000010e2 <+34>: movq 0xf17(%rip), %rdi ; (void *)0x00007fff78ba3250: std::__1::cin  0x1000010e9 <+41>: leaq -0x8(%rbp), %rsi  0x1000010ed <+45>: movq %rax, -0x10(%rbp)  0x1000010f1 <+49>: callq 0x100001d74 ; symbol stub for: std::__1::basic_istream<char, std::__1::char_traits<char> >::operator>>(int&)  0x1000010f6 <+54>: movl -0x8(%rbp), %edi  0x1000010f9 <+57>: movq %rax, -0x18(%rbp)  0x1000010fd <+61>: callq 0x1000011a0 ; ___lldb_unnamed_symbol3$$verify  0x100001102 <+66>: testb $0x1, %al  0x100001104 <+68>: jne 0x10000110f ; <+79>  0x10000110a <+74>: jmp 0x10000112b ; <+107>  0x10000110f <+79>: movq 0xef2(%rip), %rdi ; (void *)0x00007fff78ba32f8: std::__1::cout  0x100001116 <+86>: leaq 0xe4f(%rip), %rsi ; "Correct!\n"  0x10000111d <+93>: callq 0x100001150 ; ___lldb_unnamed_symbol2$$verify  0x100001122 <+98>: movq %rax, -0x20(%rbp)  0x100001126 <+102>: jmp 0x100001142 ; <+130>  0x10000112b <+107>: movq 0xed6(%rip), %rdi ; (void *)0x00007fff78ba32f8: std::__1::cout  0x100001132 <+114>: leaq 0xe3d(%rip), %rsi ; "Incorrect!\n"  0x100001139 <+121>: callq 0x100001150 ; ___lldb_unnamed_symbol2$$verify  0x10000113e <+126>: movq %rax, -0x28(%rbp)  0x100001142 <+130>: xorl %eax, %eax  0x100001144 <+132>: addq $0x30, %rsp  0x100001148 <+136>: popq %rbp  0x100001149 <+137>: retq  0x10000114a <+138>: nopw (%rax,%rax) (lldb)

The verify function is called at 0x1000010fd if afterwards it jumps to 0x10000110f then the number is verified correctly, and if not it jumps to 0x10000112b. We still want it to ask for a number but completely skip the verification. The crude way is to insert NOPs from 0x1000010f6 to 0x10000110f because it will arrive straight at the part where it prints "Correct!". That means 25 NOPs.

I created the “nop.py” script to insert NOPs:

% ./nop.py verify 0x10f6 25 Patching "verify" at offset 4342 with 25 NOPs Read 14416 bytes Changing values at 4342 to 4367 Writing new data % ./verify Serial number: 83 Correct!

Thus 83 is now deemed correct!

Let’s take a look at the actual contents of the binary now:

(lldb) dis verify`___lldb_unnamed_symbol1$$verify: -> 0x1000010c0 <+0>: pushq %rbp  0x1000010c1 <+1>: movq %rsp, %rbp  0x1000010c4 <+4>: subq $0x30, %rsp  0x1000010c8 <+8>: movq 0xf39(%rip), %rdi ; (void *)0x00007fff78ba32f8: std::__1::cout  0x1000010cf <+15>: leaq 0xe86(%rip), %rsi ; "Serial number: "  0x1000010d6 <+22>: movl $0x0, -0x4(%rbp)  0x1000010dd <+29>: callq 0x100001150 ; ___lldb_unnamed_symbol2$$verify  0x1000010e2 <+34>: movq 0xf17(%rip), %rdi ; (void *)0x00007fff78ba3250: std::__1::cin  0x1000010e9 <+41>: leaq -0x8(%rbp), %rsi  0x1000010ed <+45>: movq %rax, -0x10(%rbp)  0x1000010f1 <+49>: callq 0x100001d74 ; symbol stub for: std::__1::basic_istream<char, std::__1::char_traits<char> >::operator>>(int&)  0x1000010f6 <+54>: nop  0x1000010f7 <+55>: nop  0x1000010f8 <+56>: nop  ...  0x10000110c <+76>: nop  0x10000110d <+77>: nop  0x10000110e <+78>: nop  0x10000110f <+79>: movq 0xef2(%rip), %rdi ; (void *)0x00007fff78ba32f8: std::__1::cout  0x100001116 <+86>: leaq 0xe4f(%rip), %rsi ; "Correct!\n"  0x10000111d <+93>: callq 0x100001150 ; ___lldb_unnamed_symbol2$$verify  0x100001122 <+98>: movq %rax, -0x20(%rbp)  0x100001126 <+102>: jmp 0x100001142 ; <+130>  0x10000112b <+107>: movq 0xed6(%rip), %rdi ; (void *)0x00007fff78ba32f8: std::__1::cout  0x100001132 <+114>: leaq 0xe3d(%rip), %rsi ; "Incorrect!\n"  0x100001139 <+121>: callq 0x100001150 ; ___lldb_unnamed_symbol2$$verify  0x10000113e <+126>: movq %rax, -0x28(%rbp)  0x100001142 <+130>: xorl %eax, %eax  0x100001144 <+132>: addq $0x30, %rsp  0x100001148 <+136>: popq %rbp  0x100001149 <+137>: retq  0x10000114a <+138>: nopw (%rax,%rax)

The 25 NOPs are easily spotted.

Recompile and strip the “verify” program again to get rid of the NOPs.

Another approach is to insert a jump instruction that takes us directly to 0x10000110f. We want to insert the jump at 0x1000010f6 where we have 3 bytes of room to change, so a relative jump would count from the next instruction (after inserting) to 0x10000110f. Since we are within a one-byte short jump distance of +/-128 we can use a JMP rel8 with jump instruction opcode 0xeb (for other jump instructions take a look here). This leaves one byte dangling which we will fill with a NOP. However, this effectively will change the address of the next instruction after our JMP to be 0x1000010f8. The distance from that to 0x10000110f is 0x17=23, which means we will insert the bytes eb 17 90 at 0x1000010f6.

I wrote another script, “patch_bytes.py”, that will allow us to insert those bytes:

% ./patch_bytes.py verify 0x10f6 0xeb 0x17 0x90 Patching "verify" at offset 4342 with: [235, 23, 144] (3 bytes) Read 14416 bytes Changing values at 4342 to 4345 Writing new data % ./verify Serial number: 83 Correct!

Let’s take a look at the resulting binary:

(lldb) dis -b verify`___lldb_unnamed_symbol1$$verify: -> 0x1000010c0 <+0>: 55 pushq %rbp  0x1000010c1 <+1>: 48 89 e5 movq %rsp, %rbp  ...  0x1000010f6 <+54>: eb 17 jmp 0x10000110f ; <+79>  0x1000010f8 <+56>: 90 nop  0x1000010f9 <+57>: 48 89 45 e8 movq %rax, -0x18(%rbp)  ...  0x10000110f <+79>: 48 8b 3d f2 0e 00 00 movq 0xef2(%rip), %rdi ; (void *)0x00007fff78ba32f8: std::__1::cout  0x100001116 <+86>: 48 8d 35 4f 0e 00 00 leaq 0xe4f(%rip), %rsi ; "Correct!\n"  0x10000111d <+93>: e8 2e 00 00 00 callq 0x100001150 ; ___lldb_unnamed_symbol2$$verify  0x100001122 <+98>: 48 89 45 e0 movq %rax, -0x20(%rbp)  0x100001126 <+102>: e9 17 00 00 00 jmp 0x100001142 ; <+130>  0x10000112b <+107>: 48 8b 3d d6 0e 00 00 movq 0xed6(%rip), %rdi ; (void *)0x00007fff78ba32f8: std::__1::cout  0x100001132 <+114>: 48 8d 35 3d 0e 00 00 leaq 0xe3d(%rip), %rsi ; "Incorrect!\n"  0x100001139 <+121>: e8 12 00 00 00 callq 0x100001150 ; ___lldb_unnamed_symbol2$$verify  0x10000113e <+126>: 48 89 45 d8 movq %rax, -0x28(%rbp)  0x100001142 <+130>: 31 c0 xorl %eax, %eax  0x100001144 <+132>: 48 83 c4 30 addq $0x30, %rsp  0x100001148 <+136>: 5d popq %rbp  0x100001149 <+137>: c3 retq  0x10000114a <+138>: 66 0f 1f 44 00 00 nopw (%rax,%rax)

Take a look in particular at the following to view our inserted instructions:

0x1000010f6 <+54>: eb 17 jmp 0x10000110f ; <+79> 0x1000010f8 <+56>: 90 nop

Continue to read the third part about changing values in a program, and live modification of the memory of a running program.

Patching Binaries: Strings

2016-09-05T22:31:40+02:00

Patching Binaries will be a series of articles about how to extract information and modify program behavior. It focuses on the Mac Mach-O executable format for the x86-64 architecture, but the techniques are similar for other formats.

The files used in this article can be found here.

One of the first things to look at in a binary, when trying to determine information about it, is the constant strings saved within it. In Mach-O binaries these can be found in section __cstring of segment __TEXT.

Compile the example program “test.cc” as “test” and run it:

% ./test std::string example. And a char* string, too.

To list a programs’ strings there are at least two programs to help us do that; otool and strings.

The most simple is using strings:

% strings -o -t x test 1f50 std::string example. 1f65 And a char* string, too.

It shows the strings and their hexadecimal offsets in the binary.

A little more detail can be obtained with otool though:

% otool -s __TEXT __cstring test test: Contents of (__TEXT,__cstring) section 0000000100001f50 73 74 64 3a 3a 73 74 72 69 6e 67 20 65 78 61 6d 0000000100001f60 70 6c 65 2e 00 41 6e 64 20 61 20 63 68 61 72 2a 0000000100001f70 20 73 74 72 69 6e 67 2c 20 74 6f 6f 2e 00 0a 00 % otool -v -s __TEXT __cstring test test: Contents of (__TEXT,__cstring) section 0000000100001f50 std::string example. 0000000100001f65 And a char* string, too. 0000000100001f7e \n

To understand the layout of the binary block above the strings are end-to-end:

0000000100001f50 73 74 64 3a 3a 73 74 72 69 6e 67 20 65 78 61 6d 0000000100001f60 70 6c 65 2e 00 0000000100001f65 41 6e 64 20 61 20 63 68 61 72 2a 20 73 74 72 69 0000000100001f75 6e 67 2c 20 74 6f 6f 2e 00 0000000100001f7e 0a 00

Notice the zero byte for termination at the end of each string!

We now know that there are two “interesting” strings at offset 1f50 and 1f65, which means if we read the binary’s data at those positions we will find the strings.

The next natural thing to do is modifying the strings. I wrote a python script “patch.py” to patch a binary at an offset with a new string.

Let’s change the first string to become "Hello, World!":

% ./patch.py test 0x1f50 "Hello, World! " Patching "test" at offset 8016 with: "Hello, World! " (20 bytes) Read 15392 bytes Size of string at 8016 is 20 bytes Changing values at 8016 to 8036 Writing new data % ./test Hello, World! And a char* string, too.

Voila!

Note that the size of the string we patched is 20 bytes, which means the value to overwrite it cannot be longer than that. To visually clear the old value we put spaces as padding at the end.

The contents of the binary are now changed to the following:

% otool -s __TEXT __cstring test test: Contents of (__TEXT,__cstring) section 0000000100001f50 48 65 6c 6c 6f 2c 20 57 6f 72 6c 64 21 20 20 20 0000000100001f60 20 20 20 20 00 41 6e 64 20 61 20 63 68 61 72 2a 0000000100001f70 20 73 74 72 69 6e 67 2c 20 74 6f 6f 2e 00 0a 00

Notice the 7 spaces (byte 20) after our string.

As a second example compile “password.cc” into “password” and run it:

% ./password Password: test Invalid! Password: password Invalid! Password: 1234 Invalid! Password: ^C

Okay, so maybe we can’t just outright guess it. Let’s look at the strings in the binary instead for clues:

% otool -v -s __TEXT __cstring password password: Contents of (__TEXT,__cstring) section 0000000100001f28 Password: 0000000100001f33 AK9FJ31P 0000000100001f3c You've entered the correct password!\n 0000000100001f62 Invalid!\n

One value looks curious: “AK9FJ31P”

% ./password Password: AK9FJ31P You've entered the correct password!

Keep in mind that this is a very simple example and nothing akin to a real-world example. It just illustrates the technique of finding constant strings in a binary that can be used in different ways.

Lastly we will briefly see how the offsets to the constant strings relate to the machine code instructions of the executable. We will do this by using the debugger LLDB to find the spot where the password of the previous example is loaded into memory.

We start by running LLDB and braking when the main() is called:

% lldb ./password (lldb) target create "./password" Current executable set to './password' (x86_64). (lldb) b main Breakpoint 1: where = password`main, address = 0x0000000100000650 (lldb) r Process 84336 launched: './password' (x86_64) Process 84336 stopped * thread #1: tid = 0x212d03, 0x0000000100000650 password`main, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1  frame #0: 0x0000000100000650 password`main password`main: -> 0x100000650 <+0>: pushq %rbp  0x100000651 <+1>: movq %rsp, %rbp  0x100000654 <+4>: subq $0xd0, %rsp  0x10000065b <+11>: xorl %esi, %esi

Since we do not know the address of the instructions that loads the string we have to step through the program until we find it. To get a better overview we will disassemble the main():

(lldb) dis password`main: -> 0x100000650 <+0>: pushq %rbp  0x100000651 <+1>: movq %rsp, %rbp  0x100000654 <+4>: subq $0xd0, %rsp  0x10000065b <+11>: xorl %esi, %esi  0x10000065d <+13>: movl $0x18, %eax ......  0x1000006fb <+171>: jmp 0x1000006d7 ; <+135>  0x100000700 <+176>: jmp 0x100000705 ; <+181>  0x100000705 <+181>: movq 0x18fc(%rip), %rdi ; (void *)0x00007fff78ba32f8: std::__1::cout  0x10000070c <+188>: leaq 0x1815(%rip), %rsi ; "Password: "  0x100000713 <+195>: callq 0x100000820 ; std::__1::basic_ostream<char, std::__1::char_traits<char> >& std::__1::operator<<<std::__1::char_traits<char> >(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, char const*)  0x100000718 <+200>: movq %rax, -0xa8(%rbp)  0x10000071f <+207>: jmp 0x100000724 ; <+212>  0x100000724 <+212>: movq 0x18d5(%rip), %rdi ; (void *)0x00007fff78ba3250: std::__1::cin ......  0x100000737 <+231>: movq %rax, -0xb0(%rbp)  0x10000073e <+238>: jmp 0x100000743 ; <+243>  0x100000743 <+243>: leaq 0x17e9(%rip), %rax ; "AK9FJ31P" ......

From the above excerpt it is shown that the "Password: " text is loaded at 0x10000070c, and that the input of std::cin at 0x100000724 is compared with the real password "AK9FJ31P" on 0x100000743. But let’s look closer at that last address:

(lldb) dis -b -c 1 -s 0x100000743 password`main:  0x100000743 <+243>: 48 8d 05 e9 17 00 00 leaq 0x17e9(%rip), %rax ; "AK9FJ31P"

We know that "AK9FJ31P" resides at offset 1f33 but here it uses 17e9, why? To understand this it is necessary to know that it uses relative addressing. So the leaq (load effective address) is invoked at 743 and if we add that to 17e9 then we get 1f2c. That’s still not the correct offset! However, looking at the assembly code we see that it uses 7 bytes, and now the equation fits: 743+7+17e9=1f33!

Note that we used an unstripped binary for this example, which means it still contains useful information. However, with a real-world example it will most likely be stripped of symbols.

Let’s remove the symbols:

% strip password

Now when we try to hook up on the main() it is evident that we can’t because the symbol is not known:

% lldb ./password (lldb) target create "./password" Current executable set to './password' (x86_64). (lldb) b main Breakpoint 1: no locations (pending). WARNING: Unable to resolve breakpoint to any actual locations.

Instead we will run the program until it asks for the password and then break to see a backtrace to get our bearings:

(lldb) r Process 86636 launched: './password' (x86_64) Password: Process 86636 stopped * thread #1: tid = 0x225927, 0x00007fff9218df8a libsystem_kernel.dylib`__read_nocancel + 10, stop reason = signal SIGSTOP  frame #0: 0x00007fff9218df8a libsystem_kernel.dylib`__read_nocancel + 10 libsystem_kernel.dylib`__read_nocancel: -> 0x7fff9218df8a <+10>: jae 0x7fff9218df94 ; <+20>  0x7fff9218df8c <+12>: movq %rax, %rdi  0x7fff9218df8f <+15>: jmp 0x7fff921887cd ; cerror_nocancel  0x7fff9218df94 <+20>: retq (lldb) bt * thread #1: tid = 0x225927, 0x00007fff9218df8a libsystem_kernel.dylib`__read_nocancel + 10, stop reason = signal SIGSTOP  * frame #0: 0x00007fff9218df8a libsystem_kernel.dylib`__read_nocancel + 10  frame #1: 0x00007fff8d103155 libsystem_c.dylib`_sread + 16  frame #2: 0x00007fff8d102769 libsystem_c.dylib`__srefill1 + 24  frame #3: 0x00007fff8d102884 libsystem_c.dylib`__srget + 14  frame #4: 0x00007fff8d0fe52b libsystem_c.dylib`getc + 52  frame #5: 0x00007fff8dd38051 libc++.1.dylib`std::__1::__stdinbuf<char>::__getchar(bool) + 119  frame #6: 0x00007fff8dd2ee4c libc++.1.dylib`std::__1::basic_istream<char, std::__1::char_traits<char> >::sentry::sentry(std::__1::basic_istream<char, std::__1::char_traits<char> >&, bool) + 200  frame #7: 0x000000010000089e password`___lldb_unnamed_symbol3$$password + 46  frame #8: 0x0000000100000737 password`___lldb_unnamed_symbol1$$password + 231  frame #9: 0x00007fff9a14a5ad libdyld.dylib`start + 1

The backtrace shows two function calls in our password executable. Taking a look at frame 8 shows the same location we used previously to argument about the address of the constant string being loaded:

(lldb) f 8 frame #8: 0x0000000100000737 password`___lldb_unnamed_symbol1$$password + 231 password`___lldb_unnamed_symbol1$$password:  0x100000737 <+231>: movq %rax, -0xb0(%rbp)  0x10000073e <+238>: jmp 0x100000743 ; <+243>  0x100000743 <+243>: leaq 0x17e9(%rip), %rax ; "AK9FJ31P"  0x10000074a <+250>: leaq -0x88(%rbp), %rcx

Continue to read the second part about “jumping the fence” by employing jumps and bypassing instructions.

Sigs: Ambiguous Member Functions

2016-09-04T13:02:42+02:00

I recently updated my sigs project (simple thread-safe signal/slot C++ library) with support for ambiguous member functions by use of the sigs::Use<>::overloadOf():

template <typename... Args> struct Use {  template <typename Cls, typename Ret>  static auto overloadOf(Ret (Cls::*MembFunc)(Args...)) {  return MembFunc;  } };

It makes use of variadic templates and auto return type deduction so the compiler must be C++14 compliant.

To illustrate why it was needed consider this: Sometimes there are several overloads for a given function and then it’s not enough to just specify &Class::functionName because the compiler does not know which overload to choose.

The Ambiguous class has two overloads of foo:

class Ambiguous { public:  void foo(int i, int j) { std::cout << "Ambiguous::foo(int, int)\n"; }  void foo(int i, float j) { std::cout << "Ambiguous::foo(int, float)\n"; } };

Without the new construct the following would result in compilation failure:

sigs::Signal<void(int, int)> s; Ambiguous amb; s.connect(&amb, &Ambiguous::foo); // <-- Will fail!

The error would look something similar to this:

test.cc:147:7: error: no matching member function for call to 'connect'  s.connect(&amb, &Ambiguous::foo);  ~~^~~~~~~ ./sigs.h:144:16: note: candidate template ignored: couldn't infer template argument 'MembFunc'  Connection connect(Instance *instance, MembFunc Instance::*mf) {

To fix it we must use sigs::Use<>::overloadOf():

s.connect(&amb, sigs::Use<int, int>::overloadOf(&Ambiguous::foo)); s(42, 48); /* Prints: Ambiguous::foo(int, int) */

Without changing the signal we can also connect the second overload foo(int, float):

// This one only works because int can be coerced into float! s.connect(&amb, sigs::Use<int, float>::overloadOf(&Ambiguous::foo)); s(12, 34); /* Prints: Ambiguous::foo(int, int) Ambiguous::foo(int, float) */

LLDB: Debugging Smart Pointers

2016-09-02T23:36:02+02:00

In recent years I have switched to using LLDB (of the LLVM project) as my primary debugger. In this post I’ll show how to debug instances of std::shared_ptr and std::unique_ptr, and what the differences are.

Create “pointers.cc” with the following contents:

#include <memory> class Test { public:  Test() : a(1), b(2), c(3) { }  int a, b, c; }; int main() {  auto sptr = std::make_shared<Test>();  auto uptr = std::make_unique<Test>();  return 0; }

Compile the program as pointers with debugging symbols. The compiler must be C++14 compliant like Clang 3.4+ or GCC 4.9+ due to the usage of std::make_unique. Let’s load it up with LLDB:

% lldb ./pointers (lldb) target create "./pointers" Current executable set to './pointers' (x86_64). (lldb) b 13 Breakpoint 1: where = pointers`main + 343 at pointers.cc:13, address = 0x0000000100001357 (lldb) r Process 65102 launched: './pointers' (x86_64) Process 65102 stopped * thread #1: tid = 0x18881d, 0x0000000100001357 pointers`main + 343 at pointers.cc:13, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1  frame #0: 0x0000000100001357 pointers`main + 343 at pointers.cc:13  10 int main() {  11 auto sptr = std::make_shared<Test>();  12 auto uptr = std::make_unique<Test>(); -> 13 return 0;  14 }

We want to find out what the member variables of the two pointers are. If we just print the value it won’t help much:

(lldb) p sptr (std::__1::shared_ptr<Test>) $0 = std::__1::shared_ptr<Test>::element_type @ 0x00000001001037f8 strong=1 weak=1 {  __ptr_ = 0x00000001001037f8 }

We observe that sptr has the member __ptr_ with address 0x00000001001037f8, which is how we access its members:

(lldb) p sptr.__ptr_ (element_type *) $1 = 0x00000001001037f8 (lldb) p *sptr.__ptr_ (std::__1::shared_ptr<Test>::element_type) $2 = (a = 1, b = 2, c = 3)

The ::element_type has type Test so the following is equivalent:

(lldb) p *(Test*)sptr.__ptr_ (Test) $3 = (a = 1, b = 2, c = 3)

The address can also be used directly:

(lldb) p *(Test*)0x00000001001037f8 (Test) $4 = (a = 1, b = 2, c = 3)

Now, uptr is a little different:

(lldb) p uptr (std::__1::unique_ptr<Test, std::__1::default_delete<Test> >) $5 = {  __ptr_ = {  std::__1::__libcpp_compressed_pair_imp<Test *, std::__1::default_delete<Test>, 2> = {  __first_ = 0x0000000100101490  }  } }

It also has the __ptr_ but it doesn’t point directly to its value like sptr.__ptr_ does:

(lldb) p uptr.__ptr_ (std::__1::__compressed_pair<Test *, std::__1::default_delete<Test> >) $6 = {  std::__1::__libcpp_compressed_pair_imp<Test *, std::__1::default_delete<Test>, 2> = {  __first_ = 0x0000000100101490  } } (lldb) p *uptr.__ptr_ error: indirection requires pointer operand ('std::__1::__compressed_pair<Test *, std::__1::default_delete<Test> >' invalid)

Instead we have to reference the __first_ member, but can still use the address directly, of course:

(lldb) p uptr.__ptr_.__first_ (Test *) $7 = 0x0000000100101490 (lldb) p *uptr.__ptr_.__first_ (Test) $8 = (a = 1, b = 2, c = 3) (lldb) p *(Test*)0x0000000100101490 (Test) $9 = (a = 1, b = 2, c = 3)

Proper usage of QThread

2013-03-06T16:13:10+02:00

QThread should be viewed as the mediator with the OS of the new thread and not as being the actual thread itself!

Don’t subclass QThread only to extend the actual thread implementation which is hardly ever necessary.
Create a QObject custom class that does what should be done and has a public slot for processing.
Create a QThread and custom instance, call custom->moveToThread(thread).
Connect signals from thread to custom instance, started/finished along with deleteLater() invocations.
Finally call thread->start().

AND very important to not allocate on heap in the constructor of the custom class only on the stack. Objects can be allocated on the heap when the public slot is called, and thus the context is the new thread and not the “home thread”.

Unleashing the Power of GDB (part 2)

2013-03-03T14:27:49+02:00

The GNU Debugger^[1] is my favorite debugging tool and I personally think it’s essential for any *nix developer to know how to use it properly if you’re working with C/C++, D, Go, Obj-C, Fortran, Pascal, Modula-2 or Ada^[2].

This is the second part of Unleashing the power of GDB. It is recommended reading the first part before proceeding. First section is about integrating GDB with Emacs, the second is about how to debug errors as they occur with common cases and the last section is about debugging without any debug symbols.

1. Integration with Emacs

GDB is taken to the next level when used together with my favorite editor Emacs^[3] because it adds autocompletion of commands and arguments, and easy stepping in the command history - saves a lot of time. But perhaps even more important: If the program is compiled with debugging symbols it will also load and show the source code. Thus hitting breakpoints and stepping through the program will show the placement in the source code as well.

Let’s take the first snippet of code from the previous article:

#include <string> #include <iostream> using namespace std; void func2(const string &str) {  cout << "func2 says: " << str << endl; } void func1(int i) {  string str(i, "$");  func2(str); } int main(int argc, char **argv) {  for (int i = 1; i <= 3; i++) {  func1(i * 5);  }  return 0; }

And compile it:

g++ -g -o test test.cpp

Open Emacs if you didn’t use it already to save the file. Hit M-x gdb <RET> to fire up the GDB mode. I recommend using the most recent GDB (7.5+) together with Emacs because it works better.

Give the following arguments to the GDB mode:

gdb -i=mi /path/to/program

The current working directory is the directory of the file being edited when going into GDB mode. So if editing /tmp/test.cpp then gdb -i=mi test will run /tmp/test.

Note that the above assumed Emacs version 24. If you are running previous versions you can use this instead:

gdb --annotate=3 /path/to/program

Thanks to Paw for pointing that out.

Let’s break at the main function:

(gdb) b main (gdb) r

If you see the following error while running on OSX:

Starting program: /x/y/foo Unable to find Mach task port for process-id 76352: (os/kern) failure (0x5).  (please check gdb is codesigned - see taskgated(8))

Then you might want to follow these instructions to give GDB the necessary rights for debugging (you have to sign the GDB executable).

This is how it looks with my Emacs configuration:

2. Inspecting actual bugs

Now I’ll show some typical programming bugs and how to spot and fix them using GDB. Suppose you have written a program but you get the following error when running it:

./prog1 segmentation fault ./prog1

Note that the error might be displayed in various of ways. A segmentation fault happens but you will be wondering what caused it so we fire up GDB:

gdb ./prog1 Reading symbols for shared libraries ... done (gdb) r Starting program: /private/tmp/prog1  Reading symbols for shared libraries ++............................. done Program received signal EXC_BAD_ACCESS, Could not access memory. Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000000 0x0000000100000f27 in main (argc=1, argv=0x7fff5fbffa78) at prog1.cpp:3 3 *ptr = 1; (gdb) p ptr $1 = (int *) 0x0

First it is observed that we are denied access to some memory at address 0x0..0 and that it’s the line *ptr = 1 (prog1.cpp:3) that caused it. So we already know from the error message that ptr must then be a null-pointer but we double-check by inspecting it with p ptr.

Another example might be the following:

./prog2 prog2(12673) malloc: *** error for object 0x7ffd08c03930: pointer being freed was not allocated *** set a breakpoint in malloc_error_break to debug abort ./prog2

From the error message we now know that some unallocated memory was freed. It also gives a tip on where to start debugging, so let’s do just that:

gdb ./prog2 Reading symbols for shared libraries ... done (gdb) b malloc_error_break Function "malloc_error_break" not defined. Make breakpoint pending on future shared library load? (y or [n]) y Breakpoint 1 (malloc_error_break) pending. (gdb) r Starting program: /private/tmp/prog2  Reading symbols for shared libraries ++............................. done Breakpoint 1 at 0x7fff9439b558 Pending breakpoint 1 - "malloc_error_break" resolved prog2(12693) malloc: *** error for object 0x100103920: pointer being freed was not allocated *** set a breakpoint in malloc_error_break to debug Breakpoint 1, 0x00007fff9439b558 in malloc_error_break () (gdb) bt #0 0x00007fff9439b558 in malloc_error_break () #1 0x00007fff9439c912 in free () #2 0x0000000100000f03 in main (argc=1, argv=0x7fff5fbffa78) at prog2.cpp:4 (gdb) f 2 #2 0x0000000100000f03 in main (argc=1, argv=0x7fff5fbffa78) at prog2.cpp:4 4 delete[] txt;  (gdb) l 1 int main(int argc, char **argv) { 2 char *txt = new char[128]; 3 delete[] txt; 4 delete[] txt;  5 return 0; 6 }

So we set a pending breakpoint, start the program, find the frame and observe the double delete[] on line 3 and 4.

A third possibility is a floating point exception:

./prog3 floating point exception ./prog3

Using GDB we immediately see the problem:

gdb ./prog3 Reading symbols for shared libraries ... done (gdb) r Starting program: /private/tmp/prog3  Reading symbols for shared libraries ++............................. done Program received signal EXC_ARITHMETIC, Arithmetic exception. 0x0000000100000f38 in main (argc=1, argv=0x7fff5fbffa78) at prog3.cpp:3 3 return a / 0;

Which in this case is a division-by-zero error. Pay attention to the warnings your compiler issues to avoid these things. It might have looked like this:

prog3.cpp:3: warning: division by zero in ‘a / 0’

Note that these are just examples of runtime errors. A lot more exist that I have not covered here.

3. Debugging with no debugging symbols

Sometimes the program you are debugging has no symbol table and you still have to locate the problem. It might be that you are debugging a third party binary so you don’t have access to the source code. This section will be a bit advanced.

Let’s start with the first problematic program from last section: prog1. This is what GDB tells us with no debug symbols:

gdb ./prog1 Reading symbols for shared libraries ... done (gdb) r Starting program: /private/tmp/prog1  Reading symbols for shared libraries ++............................. done Program received signal EXC_BAD_ACCESS, Could not access memory. Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000000 0x0000000100000f27 in main ()

Unfortunately we cannot inspect the source code but instead we can inspect the machine code by disassembling a section of memory into symbolic language:

(gdb) disas Dump of assembler code for function main: 0x0000000100000f10 <main+0>: push %rbp 0x0000000100000f11 <main+1>: mov %rsp,%rbp 0x0000000100000f14 <main+4>: mov %edi,-0x4(%rbp) 0x0000000100000f17 <main+7>: mov %rsi,-0x10(%rbp) 0x0000000100000f1b <main+11>: movq $0x0,-0x20(%rbp) 0x0000000100000f23 <main+19>: mov -0x20(%rbp),%rax 0x0000000100000f27 <main+23>: movl $0x1,(%rax) 0x0000000100000f2d <main+29>: movl $0x0,-0x18(%rbp) 0x0000000100000f34 <main+36>: mov -0x18(%rbp),%eax 0x0000000100000f37 <main+39>: mov %eax,-0x14(%rbp) 0x0000000100000f3a <main+42>: mov -0x14(%rbp),%eax 0x0000000100000f3d <main+45>: pop %rbp 0x0000000100000f3e <main+46>: retq  End of assembler dump.

Let us focus around the reported address 0x0000000100000f27:

(gdb) disas 0x0000000100000f1b 0x0000000100000f2d Dump of assembler code from 0x100000f1b to 0x100000f2d: 0x0000000100000f1b <main+11>: movq $0x0,-0x20(%rbp) 0x0000000100000f23 <main+19>: mov -0x20(%rbp),%rax 0x0000000100000f27 <main+23>: movl $0x1,(%rax) End of assembler dump. ```` On the third line the pointer is accessed and `1` is attempted saved at the location. So this information can be given to the third party. However, we actually know the source code so I'll show the parallels: ```shell movq $0x0,-0x20(%rbp) int *ptr = 0; mov -0x20(%rbp),%rax // Put addr 0 in %rax movl $0x1,(%rax) *ptr = 1;

On the first line 0 is saved at -0x20(%rbp), line two moves the address (0) to %rax and the third line tries to dereference the null-pointer and assign 1 to it.

Let us try to do the same thing for prog2:

gdb ./prog2 Reading symbols for shared libraries ... done (gdb) b malloc_error_break Function "malloc_error_break" not defined. Make breakpoint pending on future shared library load? (y or [n]) y Breakpoint 1 (malloc_error_break) pending. (gdb) r Starting program: /private/tmp/prog2  Reading symbols for shared libraries ++............................. done Breakpoint 1 at 0x7fff9439b558 Pending breakpoint 1 - "malloc_error_break" resolved prog2(15026) malloc: *** error for object 0x100103920: pointer being freed was not allocated *** set a breakpoint in malloc_error_break to debug Breakpoint 1, 0x00007fff9439b558 in malloc_error_break () (gdb) bt #0 0x00007fff9439b558 in malloc_error_break () #1 0x00007fff9439c912 in free () #2 0x0000000100000f03 in main ()

The exception arises on address 0x0000000100000f03, so again we have to disassemble to make sense of it:

(gdb) disas Dump of assembler code for function main: 0x0000000100000eb0 <main+0>: push %rbp 0x0000000100000eb1 <main+1>: mov %rsp,%rbp 0x0000000100000eb4 <main+4>: sub $0x20,%rsp 0x0000000100000eb8 <main+8>: mov %edi,%eax 0x0000000100000eba <main+10>: mov %eax,-0x4(%rbp) 0x0000000100000ebd <main+13>: mov %rsi,-0x10(%rbp) 0x0000000100000ec1 <main+17>: mov $0x80,%rax 0x0000000100000ecb <main+27>: mov %rax,%rdi 0x0000000100000ece <main+30>: callq 0x100000f20 <dyld_stub__Znam> 0x0000000100000ed3 <main+35>: mov %rax,-0x20(%rbp) 0x0000000100000ed7 <main+39>: mov -0x20(%rbp),%rax 0x0000000100000edb <main+43>: cmp $0x0,%rax 0x0000000100000edf <main+47>: je 0x100000eed <main+61> 0x0000000100000ee1 <main+49>: mov -0x20(%rbp),%rax 0x0000000100000ee5 <main+53>: mov %rax,%rdi 0x0000000100000ee8 <main+56>: callq 0x100000f1a <dyld_stub__ZdaPv> 0x0000000100000eed <main+61>: mov -0x20(%rbp),%rax 0x0000000100000ef1 <main+65>: cmp $0x0,%rax 0x0000000100000ef5 <main+69>: je 0x100000f03 <main+83> 0x0000000100000ef7 <main+71>: mov -0x20(%rbp),%rax 0x0000000100000efb <main+75>: mov %rax,%rdi 0x0000000100000efe <main+78>: callq 0x100000f1a <dyld_stub__ZdaPv> 0x0000000100000f03 <main+83>: movl $0x0,-0x18(%rbp) 0x0000000100000f0a <main+90>: mov -0x18(%rbp),%eax 0x0000000100000f0d <main+93>: mov %eax,-0x14(%rbp) 0x0000000100000f10 <main+96>: mov -0x14(%rbp),%eax 0x0000000100000f13 <main+99>: add $0x20,%rsp 0x0000000100000f17 <main+103>: pop %rbp 0x0000000100000f18 <main+104>: retq  End of assembler dump.

This is a large chunk but I will point out the important little details. The following line calls new[]:

0x0000000100000ece <main+30>: callq 0x100000f20 <dyld_stub__Znam>

The reason is that _Znam actually means operator new[](unsigned long). You can use the tool c++filt or this site to do demangle symbols.

And here are the double deletions:

0x0000000100000ee8 <main+56>: callq 0x100000f1a <dyld_stub__ZdaPv> 0x0000000100000efe <main+78>: callq 0x100000f1a <dyld_stub__ZdaPv>

_ZdaPv means operator delete[](void*). In both cases they are passed the same address by the following instructions:

0x0000000100000ee1 <main+49>: mov -0x20(%rbp),%rax 0x0000000100000ee5 <main+53>: mov %rax,%rdi

And here as well:

0x0000000100000ef7 <main+71>: mov -0x20(%rbp),%rax 0x0000000100000efb <main+75>: mov %rax,%rdi

The third program with the floating-point exception is debugged in the following way without symbols:

gdb ./prog3 Reading symbols for shared libraries ... done (gdb) r Starting program: /private/tmp/prog3  Reading symbols for shared libraries ++............................. done Program received signal EXC_ARITHMETIC, Arithmetic exception. 0x0000000100000f38 in main ()

Once again it is necessary to disassemble:

(gdb) disas Dump of assembler code for function main: 0x0000000100000f20 <main+0>: push %rbp 0x0000000100000f21 <main+1>: mov %rsp,%rbp 0x0000000100000f24 <main+4>: mov %edi,-0x4(%rbp) 0x0000000100000f27 <main+7>: mov %rsi,-0x10(%rbp) 0x0000000100000f2b <main+11>: movl $0xff,-0x1c(%rbp) 0x0000000100000f32 <main+18>: mov -0x1c(%rbp),%eax 0x0000000100000f35 <main+21>: xor %ecx,%ecx 0x0000000100000f37 <main+23>: cltd  0x0000000100000f38 <main+24>: idiv %ecx 0x0000000100000f3a <main+26>: mov %eax,-0x18(%rbp) 0x0000000100000f3d <main+29>: mov %eax,-0x14(%rbp) 0x0000000100000f40 <main+32>: mov -0x14(%rbp),%eax 0x0000000100000f43 <main+35>: pop %rbp 0x0000000100000f44 <main+36>: retq  End of assembler dump.

The interesting lines are these with the exception happening on the last line:

0x0000000100000f2b <main+11>: movl $0xff,-0x1c(%rbp) 0x0000000100000f32 <main+18>: mov -0x1c(%rbp),%eax 0x0000000100000f35 <main+21>: xor %ecx,%ecx 0x0000000100000f37 <main+23>: cltd  0x0000000100000f38 <main+24>: idiv %ecx

This basically does the operation 0xFF / 0. 0xFF is saved to -0x1c(%rbp) and moved to %eax, then 0 is “created” by doing an XOR on %ecx with itself (recall that XOR on two equal operands always yields 0) and storing in %ecx, cltd converts signed long word %eax to a double word in %edx:%eax and finally idiv does the signed division.

Stay tuned for more.

Unleashing the Power of GDB

2013-02-12T01:33:30+02:00

This is the first in hopefully a series of entries about utilising GDB to its fullest. First a little about how to invoke GDB, secondly some basic usage of some of the most frequently used commands, how to attach to a running program and modify/redirect its stdout and stderr, then how to install and use watchpoints, and finally how to manually call functions and class methods.

1. Invocation

Here is how to start debugging a program:

gdb ./program

Make sure to compile your programs with debugging symbols to make source code and line numbers available while debugging. If using GCC or Clang use the -g argument to do this.

Sometimes arguments have to be passed to the program to debug:

gdb --args ./program arg1 arg2 arg3..

2. Basic usage

The basic commands to use during the debugging phase is run (r), step (s), next (n), continue (c), frame (f), backtrace (bt), print (p), info (i), break(b), lines (l), help (h) and quit (q).

run will run the program or re-run if already started.
step steps until the program reaches a new source code line and can be used with an integer argument to step that many times.
next works like `step` except it will treat a function call like a single instruction. Can also be given an integer argument.
continue will resume execution after hitting a breakpoint, for instance.
frame can be used to select or print current stack frame. When no argument is given it will print the current stack frame, and otherwise it will select the chosen frame.
backtrace is used to print a backtrace of all stack frames in order. When an argument is given it will print the innermost frames if positive and the outermost if negative.
print prints the value of an expression. It could be variables in the current stack frame, globals or an entire file. As we shall see later on it can also be used to invoke system calls and more.
info is very useful because it can display a lot of information about variables, threads, frame, registers, files, types, symbols, locals, addresses and tons more.
break is used to set breakpoints in the program at a specified line, function or address.
lines with no arguments shows the 10 source code lines centered at the current line. The argument can be a line number, a file with line number (FILE:LINENUM), a function, a file and function (FILE:FUNCTION) or an address (*ADDR).
help is invaluable because it explains about more or less everything in GDB. So use it!
Lastly quit exits the program and terminates GDB.

So let’s give them a spin. Save the following code to a file named “test.cpp”:

#include <string> #include <iostream> using namespace std; void func2(const string &str) {  cout << "func2 says: " << str << endl; } void func1(int i) {  string str(i, "$");  func2(str); } int main(int argc, char **argv) {  for (int i = 1; i <= 3; i++) {  func1(i * 5);  }  return 0; }

Then compile it thusly:

g++ -g -o test test.cpp

And debug it:

gdb ./test

First thing we will do is set a breakpoint at main and run the program:

(gdb) b main Breakpoint 1 at 0x100000c4f: file test.cpp, line 15. (gdb) r Starting program: /private/tmp/test  Reading symbols for shared libraries ++............................. done Breakpoint 1, main (argc=1, argv=0x7fff5fbffa80) at test.cpp:15 15 for (int i = 1; i <= 3; i++) { (gdb)

Then we inspect the arguments of the current stack:

(gdb) f #0 main (argc=1, argv=0x7fff5fbffa80) at test.cpp:15 15 for (int i = 1; i <= 3; i++) { (gdb) i args argc = 1 argv = (char **) 0x7fff5fbffa80 (gdb) p argv $1 = (char **) 0x7fff5fbffa80 (gdb) p *argv $2 = 0x7fff5fbffbf8 "/private/tmp/test" (gdb) p argv[0] $3 = 0x7fff5fbffbf8 "/private/tmp/test" (gdb) p argv[1] $4 = 0x0 (gdb)

Notice that the manipulation of the argv array is very much like in C/C++ with deference/element access (p *argv and p argv[1]).

Now we step one round through the for-loop:

(gdb) s 16 func1(i * 5); (gdb) s func1 (i=5) at test.cpp:10 10 string str(i, "$"); (gdb) s 11 func2(str); (gdb) s func2 (str=@0x7fff5fbffa00) at test.cpp:6 6 cout << "func2 says: " << str << endl; (gdb) s func2 says: $$$$$ 7 } (gdb) s func1 (i=5) at test.cpp:12 12 } (gdb) s 0x0000000100000bee 11 func2(str); (gdb) s main (argc=1, argv=0x7fff5fbffa80) at test.cpp:15 15 for (int i = 1; i <= 3; i++) { (gdb)

What happens is that we first call func1(5) which in turn calls func2(str) and returns to the for-loop. If we instead used next the output would be shorter because it would not step into func1:

(gdb) n 16 func1(i * 5); (gdb) n func2 says: $$$$$$$$$$ 15 for (int i = 1; i <= 3; i++) { (gdb)

It’s time to see a backtrace in action:

(gdb) b func2 Breakpoint 2 at 0x100000abc: file test.cpp, line 6. (gdb) c Continuing. Breakpoint 2, func2 (str=@0x7fff5fbffa00) at test.cpp:6 6 cout << "func2 says: " << str << endl; (gdb) bt #0 func2 (str=@0x7fff5fbffa00) at test.cpp:6 #1 0x0000000100000b81 in func1 (i=15) at test.cpp:11 #2 0x0000000100000c65 in main (argc=1, argv=0x7fff5fbffa80) at test.cpp:16 (gdb)

The trace shows the stack frames in a numbered fashion from func2 and backwards. Let’s inspect frame #1:

(gdb) f 1 #1 0x0000000100000b81 in func1 (i=15) at test.cpp:11 11 func2(str); (gdb) l 6 cout << "func2 says: " << str << endl; 7 } 8  9 void func1(int i) { 10 string str(i, "$"); 11 func2(str); 12 } 13  14 int main(int argc, char **argv) { 15 for (int i = 1; i <= 3; i++) { (gdb)

That concludes the basics:

(gdb) q The program is running. Exit anyway? (y or n) y

3. Attaching live programs

GDB can attach to running programs by stating its process ID (PID):

gdb -p PID

Here is a little trick to debug a program started by another program at runtime. It might be crucial to attach and debug “from the top” so what to do? Simply add a little sleep to leave enough time to find the PID and attach using GDB. After attaching set the needed breakpoints and continue execution. Another way of achieving the same thing is to use:

gdb --waitfor=PROCNAME

Where PROCNAME is the process name to continuously poll for until it has been launched. Note that some instructions will have been executed before GDB attaches this way because polling is not exactly instant however close it might seem to be.

A very useful technique is to know how to redirect stdout (file descriptor 1) and/or stderr (FD 2) after a program has started. We are going to exploit the fact that print can invoke system calls like dup2 and open in our case. After attaching to the process do the following:

(gdb) p (void) dup2((int) open("/tmp/out.txt", 0x201, 0640), 1) (gdb) p (void) dup2((int) open("/tmp/err.txt", 0x201, 0640), 2) (gdb) detach (gdb) q

In short it will redirect stdout to “/tmp/stdout.txt” and stderr to “/tmp/stderr.txt”. The system call open is used to open a file for writing in our case. The mode “0x201” actually means “write only and create file if nonexistent” since O_CREAT | O_WRONLY = 0x200 | 0x1 = 0x201 (see the fcntl.h header file for details). “0640” is the umask (user has RW and group has R). After opening a file and retrieving its FD we need to redirect the device in question to it (stdout and stderr in this case). This is achieved using dup2 that creates an alias to the FD, does redirection and closes the old FD.

Additionally, it’s important to cast types to enforce GDB to behave correctly. If this is not the case it could argue giving the following error message:

Unable to call function “open” at 0x7fff906e3fe4: no return type information available. To call this function anyway, you can cast the return type explicitly (e.g. ”print (float) fabs (3.0)”)

Another scenario would be to completely turn off output to stdout/stderr:

(gdb) p (void) close(1) (gdb) p (void) close(2) (gdb) det (gdb) q

Here is some code for the testing the above instructions and to observe that the output is redirected or stopped:

#include <unistd.h> #include <iostream> using namespace std; int main(int argc, char **argv) {  for (int i = 0; i < 120; i++) {  cout << "hello stdout" << endl;  cerr << "hello stderr" << endl;  sleep(2);  }  return 0; }

4. Watchpoints

Watchpoints are useful when certain variables need to be watched using the command watch. Whenever a watched variable is changed GDB will show the old and new value along with the stack frame. Try debugging the program from before:

(gdb) b main Breakpoint 1 at 0x100000c4f: file test.cpp, line 15. (gdb) r  Starting program: /private/tmp/test  Reading symbols for shared libraries ++............................. done Breakpoint 1, main (argc=1, argv=0x7fff5fbffa80) at test.cpp:15 15 for (int i = 1; i <= 3; i++) { (gdb) watch i Hardware watchpoint 2: i (gdb) d 1 (gdb) c Continuing. Hardware watchpoint 2: i Old value = 0 New value = 1 0x0000000100000c56 in main (argc=1, argv=0x7fff5fbffa80) at test.cpp:15 15 for (int i = 1; i <= 3; i++) { (gdb) c Continuing. $$$$$ Hardware watchpoint 2: i Old value = 1 New value = 2 0x0000000100000c6e in main (argc=1, argv=0x7fff5fbffa80) at test.cpp:15 15 for (int i = 1; i <= 3; i++) { (gdb) c Continuing. $$$$$$$$$$ Hardware watchpoint 2: i Old value = 2 New value = 3 0x0000000100000c6e in main (argc=1, argv=0x7fff5fbffa80) at test.cpp:15 15 for (int i = 1; i <= 3; i++) { (gdb) c Continuing. $$$$$$$$$$$$$$$ Hardware watchpoint 2: i Old value = 3 New value = 4 0x0000000100000c6e in main (argc=1, argv=0x7fff5fbffa80) at test.cpp:15 15 for (int i = 1; i <= 3; i++) { (gdb) c Continuing. Watchpoint 2 deleted because the program has left the block in which its expression is valid.

Notice I did d 1 which means to delete the breakpoint with number “1”.

5. Calling functions

While debugging it is possible to call functions of the program - even pass them arguments - and have the result saved in value history if non-void. Load the first test program from above in GDB:

(gdb) b 11 Breakpoint 1 at 0x100000b45: file test.cpp, line 11. (gdb) r Starting program: /private/tmp/test  Reading symbols for shared libraries ++............................. done Breakpoint 1, func1 (i=5) at test.cpp:11 11 func2(str); (gdb) i loc  str = {  _M_dataplus = {  <std::allocator<char>> = {  <__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>},   members of std::string::_Alloc_hider:   _M_p = 0x100103938 "$$$$$"  } } (gdb) call func2(str) func2 says: $$$$$ (gdb) q The program is running. Exit anyway? (y or n) y

Since we already have a std::string object we can call func2(str) manually.

This functionality opens up for a variety of different uses. One is to use predefined debugging functions to display custom data structures that GDB doesn’t know about. Take the std::string structure above for str - it’s not very descriptive except for the fact that its content is visible (_M_p = 0x100103938 "$$$$$").

Compile and load up the following program in GDB:

#include <string> #include <iostream> using namespace std; class Test { public:  Test() : a(1), b(2), c(3), str("Hello, World!") { }  int a, b, c;  string str; }; void dbg(const Test &test) {  cout << "a = " << test.a << endl  << "b = " << test.b << endl  << "c = " << test.c << endl  << "str = " << test.str << endl; } int main(int argc, char **argv) {  Test test;  return 0; }

Then it’s high time to compare the two approaches:

(gdb) b main Breakpoint 1 at 0x100000b31: file custom.cpp, line 21. (gdb) r Starting program: /private/tmp/custom  Reading symbols for shared libraries ++............................. done Breakpoint 1, main (argc=1, argv=0x7fff5fbffa78) at custom.cpp:21 21 Test test; (gdb) n 22 return 0; (gdb) call dbg(test) a = 1 b = 2 c = 3 str = Hello, World! (gdb) p test $1 = {  a = 1,   b = 2,   c = 3,   str = {  _M_dataplus = {  <std::allocator<char>> = {  <__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>},   members of std::string::_Alloc_hider:   _M_p = 0x100103938 "Hello, World!"  }  } } (gdb)

Keep in mind that this is just a simple test. The great thing about this is that you can fine-tune the debugging function to show only what you need to see and leave out the unnecessary parts. And make sure the variables you’re accessing are actually live and initialised when doing so! If we skip the next command above this is the result:

(gdb) call dbg(test) a = 0 b = 0 c = 0 Program received signal EXC_BAD_ACCESS, Could not access memory. Reason: KERN_INVALID_ADDRESS at address: 0xffffffffffffffe8 0x00007fff8fa68c85 in std::operator<< <char, std::char_traits<char>, std::allocator<char> > ()

It is also possible to call class methods. Add the following snippet after the constructor on line 7 in the previous code:

void __attribute__ ((used)) dump() {  cout << "str = " << str << endl; }

Notice the use of the “used” attribute - this is because if the method is not called in the source code, which is the case here, the compiler will optimise the whole thing away in most cases.

Invoking the method on our test object is easy:

(gdb) call test.dump() str = Hello, World!

That concludes the basic knowledge needed to start debugging using GDB. Stay tuned for more!

sed tricks

2013-02-09T16:05:04+02:00

The stream editor, most commonly know as sed, is a wonderful tool for modifying data from files and stdin. In this article I will be using the BSD variant of sed which is a little bit different from the GNU variant of sed but I will try to point out where the differences are in my examples.

1. Usage

One of the most common ways of using sed is:

cat file.txt | sed COMMAND

And the other

sed COMMAND FILE

It is possible to have multiple commands using the -e argument:

sed -e COMMAND -e COMMAND ..

Or even like this:

sed 'COMMAND;COMMAND;..'

The COMMAND can be a lot of different things but usually it will be the substitution pattern: s/regular expression/replacement/flags

Here is an example:

echo "aabb" | sed -e 's/a/A/' -e 's/b/B/g' AaBB

Notice that the first command is only run on the first occurrence and the second is run on all using the g flag. Keep in mind that it works on a line-by-line basis.

Another important aspect is in-place alteration of files:

sed -i (EXT) COMMAND FILE

This command will edit FILE using COMMAND and if the extension EXT is given then a backup is saved to the FILE with EXT appended to the filename. It is generally recommended to produce backup files so nothing is lost unintentionally.

2. Group matching

Often it is necessary to match some block of data and substitute some portions without removing other parts, or to alter the order of blocks. A group is matched using parentheses in the command and referenced with \\1,\\2.. etc., for instance:

echo "foobar" | sed 's/\\(foo\\)\\(bar\\)/\\2\\1/' barfoo

Two groups are matched (“foo” and “bar”) and their order is reversed. Notice that the parentheses have to be escaped in order to prevent matching the actual characters “(” and “)”.

3. Extended regular expressions

The expressions I have used up until now were basic regular expressions but a more powerful variant exists, namely the extended regular expressions. In this mode, along with a lot of stuff, it’s not necessary to escape parentheses and the POSIX character sets are available. Example:

echo "foo \\t bar" | sed -E 's/(foo)[[:space:]]*(bar)/\\2 \\1/' bar foo

The character set [:space:] matches any whitespace character. -E is replaced with -r in GNU sed.

Here is a great reference of the different regular expressions, both basic and extended ones.

4. Case sensitivity

In GNU sed there is a flag to turn off case sensitivity, namely the “i” flag. Sadly this flag is not available in BSD sed so one has to turn to other possibilities. One is to decapitalize all letters before piping to sed, but that doesn’t work for files and in-place modifications. However, case can be ignored when using character sets and similar constructs so keep that in mind, i.e. [:alpha:] will match “A” as well as “a”. Selective parts of a regular expression can sometimes require specific characters so if one wants to match both cases it can be done using ranges, like [a-cA-C] for instance.

Different approaches exist for decapitalizing using pipes. Here is one using tr:

echo "Hello, World" | tr '[:upper:]' '[:lower:]' hello, world

And here is one using awk:

echo "Hello, World" | awk '{print tolower($0)}' hello, world

If all else fails Perl has a sed-like substitution syntax that accepts the “i” flag:

echo "HeLlo" | perl -pe 's/l/./ig' He..o

5. Greedy vs non-greedy matching

sed is greedy by default, meaning it will try to match as much as possible when using + or *. Here’s a greedy example:

echo "aaa(bbb)aaa(bbb)aaa" | sed -E 's/\\(.*\\)/./' aaa.aaa

The above matches a “(” then anything greedily until next “)”. Suppose we wanted to only match the first “(bbb)” and not “(bbb)aaa(bbb)”, then we could do the following:

echo "aaa(bbb)aaa(bbb)aaa" | sed -E 's/\\([^\\(]*\\)/./' aaa.aaa(bbb)aaa

As before it matches a “(” then matching non-greedily for “(” (meaning not matching a “(“) until we match the closing “)”.

6. Examples

A friend of mine recently asked me how to replace entries of the form “<email address>” with “<—removed—>” using sed. The following was the solution:

cat file.txt | sed -E 's/(<).*@.*(>)/\\1--removed--\\2/g'

It’s not a correct regexp for matching email addresses but given the knowledge of the presence of <, > and @ it was fitting in his scenario.

The proper way of doing it would be the following:

cat file.txt | sed -E 's/(<)[[:alpha:][:digit:]\\._%\\+-]+@[[:alpha:][:digit:]\\.-]+\\.[[:alpha:]]{2,4}(>)/\\1--removed--\\2/g'

Another useful thing is to escape spaces in filenames and use them with commands:

#!/bin/sh DIR=`echo $1 | sed 's/ /\\\\\\\\ /g'` eval echo "Listing ${DIR}:" eval ls -l ${DIR}

I used double-escaped backslashes because the final output should be, for instance, “/test\ one\ seven/foobar.txt” and not “/test one seven/foobar.txt” so that the commands can interpret the paths correctly (using the eval command). The effect is that the script can be called with both an escaped or non-escaped path, i.e. "/test one seven" or /test\\ one\\ seven.

When creating a Linux distribution of a program or similar that is required to run off-the-bat, and where the source code might not be available, it is often needed to get all the shared library dependencies of certain binary files. The following script will copy these to the destination directory: (Linux only!)

#!/bin/sh # Usage: getshared.sh <binary> <dest directory> BIN=`echo $1 | sed 's/ /\\\\\\\\ /g'` DST=`echo $2 | sed 's/ /\\\\\\\\ /g'` eval mkdir -p ${DST} eval ldd ${BIN} | grep "=>" | sed 's/.*\\s*=>\\s*\\(.*\\)(.*)/\\1/' | awk '{print $1};' | eval xargs -I{} cp -vuL {} ${DST}

Here the sed command retrieves the dependency library from the output using the \\1 group. A sample line from ldd could be

libc.so.6 => /usr/x86_64-linux-gnu/libc.so.6 (0x00007f6eb8c000)

Assume we have a file we want to modify in-place, like an INI configuration file like this:

[Section] enableStuff=false  someVar=1337 enableOther = false dontTouchThis="wicked" oldSchool: false

And let’s say we wanted to replace all instances of “false” with “true”:

sed -i .bck -E 's/([[:alnum:]]+)[^\\s]*([=:])[^\\s]*false/\\1\\2true/' conf.ini

You will get the following result in conf.ini (along with the backup in conf.ini.bck):

[Section] enableStuff=true  someVar=1337 enableOther=true dontTouchThis="wicked" oldSchool:true

Notice that I remove all whitespace and that I allow both “=” and “:” as the delimiter because some implementations use “:” however uncommon it may be.

That concludes my tips and tricks using sed.

Upgrading to Xcode 4.4 or later

2013-02-03T22:13:56+02:00

Xcode moved to the App Store as of version 4.4 and it had certain consequences to the common programmer. First of all, the SDKs are no longer located at “/Developer/SDKs” like they were before. The app is located at “/Applications/Xcode.app” on a default install and it contains the SDKs in the directory:

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs

All command-line tools are no longer installed per default either so in order to install them open Xcode, go to “Preferences”, select the “Downloads”, choose “Components” and install “Command Line Tools”.

In order to configure the BSD tools correctly it is needed to instruct where Xcode is installed. It is done like this:

xcode-select --switch /Applications/Xcode.app/Contents/Developer

Another important change of Xcode 4.4+ is the discontinuation of the 10.6 SDK. When installing, or updating, 4.4+ it will only have SDKs 10.7 and 10.8. If 10.6 is installed it will be removed. So if you have 4.3 or earlier installed I recommend that you backup 10.6 if you need it for development. The SDK contains a lot of symbolic links so it’s important to not follow them:

zip -r9 --symlink /path/to/store/sdk-10.6.zip /Developer/SDKs/MacOSX10.6.sdk

If you don’t have SDK 10.6 and want it you can follow these steps:

Go to https://developer.apple.com/downloads/index.action
You have to login which thus requires an Apple Developer account.
Filter the search with “Xcode 4 and iOS SDK 4.3”.
Download “Xcode 4 and iOS SDK 4.3 - Final” DMG (4.28 GB).
Open the DMG to mount it and go to “/Volumes/Xcode and iOS SDK/Packages”.
You will find the file “MacOSX10.6.pkg” there which contains the SDK.

I personally use SDK 10.6 on a daily basis for work so I have to have it installed at all times. And since Xcode will remove it if put into its SDK directory on update I have instead created the old directory “/Developer/SDKs” and put the SDK there. Then I put symbolic links to the new SDKs 10.7 and 10.8.

Profiling with Instruments.app

2013-02-02T23:23:13+02:00

Profiling binaries is essential in any serious development process. Unfortunately not very many good profilers exist out there. I usually employ Valgrind to detect memory leaks and the front-end KCachegrind to inspect call stacks. But KCachegrind requires both X11 and KDE3 libs which I want to avoid. However, last week I discovered Instruments.app that comes with Xcode. It’s a great profiler with lots of different modes, like detecting memory leaks, tracking heap allocations, detection of “zombie” objects, time-based profiling, performance monitor counter (PMC) based sampling, event-based sampling, file activity, sudden termination analysis and much more. But Instruments.app is not good at finding uninitialized variables like Valgrind is among other things - so this is not a direct substitution!

The app is located here:

/Applications/Xcode.app/Contents/Applications/Instruments.app

Let’s write a little program with a memory leak that we want the program to detect for us.

Save the following code to a file named “memleak.cpp”:

#include <unistd.h> int main(int argc, char **argv) {  for (int i = 0; i < 10; i++) {  char *test = new char[1024];  sleep(1);  }  return 0; }

Compile the program and run Instruments.app. When it starts select the “Leaks” template in the “Mac OS X” section and click “Choose”. A new window opens up showing two instruments; one for allocations and one for explicit memory leaks. Before one can record anything the program to profile has to be chosen in the “Choose Target” drop-down menu. Click “Record” and wait for the program to terminate (after 10 seconds) and you’ll have the following results:

Clicking the “Leaks” instrument you will see:

Just to show the difference try inserting delete[] test; after line 5 in the code, recompile and re-record the analysis. You’ll see the following:

Sometimes it’s necessary and insightful to gaze upon the call stacks:

Furthermore, any system libraries and missing symbols can be hidden in the “Call Tree” sidebar for convenience. If the program was compiled with debugging symbols (use -g with GCC or Clang, for instance) the source code along with the machine code can be shown by double-clicking an entry:

In any case, I will try using this profiler more at work and see if I find more interesting features to share.

Morten's Dev

Emacs and the Language Server Protocol

1. Prerequisites

2. Emacs configuration

2.1. C++

2.2. Python

2.3. Rust

3. Usage

4. Extras

4.1. lsp-ui

4.2. hydra

4.3. company-lsp

Sigs: Migrating to C++17

1. CMake configuration changes

2. C++17 features

2.1. [[nodiscard]]

2.2. Initializer statements

2.3. std::optional

2.4. std::is_void_v

2.5. noexcept

Sigs: Return Values

Revamped and new Domain

Patching Binaries: Live Modification

Patching Binaries: Jumping The Fence

Patching Binaries: Strings

Sigs: Ambiguous Member Functions

LLDB: Debugging Smart Pointers

Proper usage of QThread

Unleashing the Power of GDB (part 2)

1. Integration with Emacs

2. Inspecting actual bugs

3. Debugging with no debugging symbols

Unleashing the Power of GDB

1. Invocation

2. Basic usage

3. Attaching live programs

4. Watchpoints

5. Calling functions

sed tricks

1. Usage

2. Group matching

3. Extended regular expressions

4. Case sensitivity

5. Greedy vs non-greedy matching

6. Examples

Upgrading to Xcode 4.4 or later

Profiling with Instruments.app

4.1. `lsp-ui`

4.2. `hydra`

4.3. `company-lsp`