Reverse Engineering

Building IDAPython on Windows

elias IDAPython, Reverse Engineering

Introduction


If you use IDAPython a lot, I am sure you started reading its source code or felt the need to add missing functionality. In this article, we will show you how to build IDAPython on Windows.

On Linux/macOS, the process is straightforward as per the BUILDING.txt.

In this article:

Make sure you already installed the latest IDA for Windows and grabbed yourself a copy of the SDK.

Let’s get started.

 

Setting up VS 2019 Community Edition


IDA SDK’s default configuration is targeted towards VS 2019. Just to keep this article simple, let’s stick with that.

Grab VS2019 from here https://visualstudio.microsoft.com/vs/older-downloads/

When you install it, make sure you leave all the default installation location (in the C:\Program Files (x86), etc.)

 

Setting up Cygwin 64


Download Cygwin 64 and install the following components:

  • make from the Devel category.
  • unzip from the Archive category.

 

Setting up Python 3.x


Download Python 3.x AMD64 (say 3.4 to 3.11) and select custom install:

  • Install to C:(X = major, Y = minor)
  • Make sure you select the following options:
    • for all users
    • pip
    • Precompile standard library
    • Optional: “Download debugging symbols”

After Python is installed, install the six module:

C:\PythonXY\Scripts\pip install six

 

Setting up SWIG


In this section, we will build SWIG for Windows from scratch. We need a special patched version of SWIG (version 4.0.1, with support for -py3-limited-api) and we cannot use the pre-built binaries.

Whether you are using a Linux distro (for example Ubuntu 20) or WSL on Windows 10/11, all the steps below still apply.

Depending on your Linux setup, you may have already installed the needed packages. Just to be on the safe side, you need the following:

sudo apt update
sudo apt install wget build-essential mingw-w64 byacc bison automake autotools-dev patchelf -y

If mingw-64 failed to install:
sudo apt-add-repository ppa:mingw-w64/ppa && sudo apt-get update && sudo apt-get install mingw-w64

Clone IDAPython’s patched SWIG

git clone --branch py3-stable-abi https://github.com/idapython/swig.git swig-py3-stable-abi

Download PCRE library and build it

Download PCRE directly into SWIG’s source directory:

wget https://master.dl.sourceforge.net/project/pcre/pcre/8.10/pcre-8.10.tar.bz2

Then edit the PCRE build script in SWIG to specify the host compiler:

Open ./Tools/pcre-build.sh and change this line:

cd pcre && ./configure --prefix=$pcre_install_dir --disable-shared $* || bail "PCRE configure failed"

To:

cd pcre && ./configure --host=x86_64-w64-mingw32 --prefix=$pcre_install_dir --disable-shared $* || bail "PCRE configure failed"

(we only added the --host switch)

Now just run the PCRE build script again:

./Tools/pcre-build.sh

You are now ready to build SWIG.

Building SWIG

First run ./autogen.sh, then run the configure script with the --host switch, while also specifying static linking:

LDFLAGS="-static -static-libgcc -static-libstdc++" \
./configure --host=x86_64-w64-mingw32 --prefix=/tmp/swig4-win

Now you can run make and make install as usual.

Alternatively, you can download the pre-build version from here.

Copy the SWIG Windows binaries from /tmp/swig4-win to your Windows machine. Let’s put them in C:\idasdk\swig4.

 

Setting up the IDA SDK

  • Download the IDA SDK and unzip to a folder, for example c:\idasdk.
  • If you have the Decompiler installed, then copy the Decompiler headers from <ida_install>/plugins/hexrays_sdk/include/hexrays.hpp to c:\idasdk\include.

Initialize the SDK’s config files

We need to configure various configurations. Open the “Developer Command Prompt for VS 2019”.

If Cygwin was not in the path, then typing ‘make’ will cause an error. If that’s the case, just add it to the PATH:

set PATH=c:\cygwin64\bin;%PATH%

(Keep that command prompt open for the remainder of this article.)

From c:\idasdk, type the following commands:

cd c:\idasdk
set __NT__=1
set __X64__=1

Now let’s generate the various config files:

  • ida64 debug build: cmd /c "set __EA64__=1 && make env"
  • ida64 optimized build: cmd /c "set __EA64__=1 && set NDEBUG=1 && make env"
  • ida debug build: make env
  • ida optimized build: cmd /c "set NDEBUG=1 && make env

If you have done everything right, you should have these files in c:\idasdk:

  • vs19paths.cfg
  • x64_win_vc_32.cfg
  • x64_win_vc_32_opt.cfg
  • x64_win_vc_64.cfg
  • x64_win_vc_64_opt.cfg

To test if everything is okay, let’s try building the hello plugin:

cd c:\idasdk\plugins\hello
make

Last but not least, let’s put the IDA binaries in the correct place in accordance with the SDK make system. Assuming that IDA was installed in C:\Tools\IDA82:

xcopy /s c:\Tools\IDA82 c:\idasdk\bin

Now, anything we build will go to c:\idasdk\bin\[plugins|loaders|procs]:

  • plugins: for plugin binaries
  • loaders: for loader modules
  • procs: for processor modules

 

Building IDAPython


Okay, now we are ready to build IDAPython!

Clone IDAPython

Navigate to c:\idasdk\plugins and clone IDAPython there:

git clone https://github.com/idapython/src.git idapython

Building IDAPython

Now we have all the prerequisit steps completed, let’s build IDAPython:

set PYTHON_VERSION_MAJOR=X
set PYTHON_VERSION_MINOR=Y

C:\PythonXY\python.exe build.py --with-hexrays --swig-home c:\ida\swig4 --ida-install c:\idasdk\bin --debug
  • Replace X and Y with the proper values
  • If you don’t have the Decompiler then omit the --with-hexrays switch
  • Similarily, drop the --debug for optimized builds.
  • Run build.py --help for more information

Be patient, this can take between 5 and 30 minutes to complete the first time.

That’s it!


I know, this has been tedious but I hope it is helpful.

Contributions and suggestions to IDAPython. We are happy to discuss and merge your changes as applicable.

climacros – IDA productivity tool

elias IDA Pro, Reverse Engineering

Introduction

A few weeks ago, I proposed an IDA features to improve the CLI and add macros support. After a few email exchanges with Arnaud from Hex-Rays, we could not agree on how to best do it and still accommodate to everyone’s needs. Finally, Arnaud kindly wrote a quick IDAPython script to show me how I can intercept the CLI text before it gets executed. Please see Hack of the day #2: Command-Line Interface helpers.

Unfortunately, the QT approach did not work for me due to many reasons:

  • I don’t want the text to be replaced on the fly.
  • I don’t want the expanded text (macros) to be part of the history. Instead the macros should remain un-expanded as they get stored in the history.

The biggest issue we were discussing is how to format numbers: 0xHEXNUM, HEXNUM, #HEXNUM, …?

How many hardcoded macros should we have and what should they do?

In all cases, since IDA is highly programmable and I have access to the awesome IDA support team (thank you guys!), the ball is now in my court and all I had to do is write a plugin.

Enter ‘climacros’

I woke up one day with the solution to this dilemma. I don’t have to hardcode anything, instead just let the users define their own macros backed by Python expressions. The only thing I hardcoded was how to embed an expression in the CLI text. I opted for something simple inspired by the PHP/ASP syntax, something like ${ expression }$ (anything between those special tags is evaluated in Python).

With this basic idea, I set about implementing all the default macros that I think are useful for my daily work with WinDbg’s CLI.

This is climacros in action:

No more copy/paste in order to get the current address from the disassembly view! 🙂

climacros is written in C++ and is opensource. You can also grab the Windows binaries from here.

Future work

If this plugin is well received, I think people will ask for features like:

  • Macros profiles: to have a set of macros that one can switch between them quickly
  • Import/Export macros: to be able to transfer macros with users
  • Macro expansion using a scripting language of your choice and not just Python

Let me know!

 

QScripts – IDA Scripting productivity tool

elias IDA Pro, Reverse Engineering

Just a quick post to introduce QScripts. QScripts is a productivity tool that helps IDA users speed up their scripts development. The idea for QScripts sprang from my autox script that I wrote for the Batchography book.

With QScripts, you will be able to develop your scripts in your favorite text editor, save the script and see the results in IDA. Therefore, it makes sense to resize IDA and the text editor to see things side by side.

Please grab your copy from here.

Daenerys: IDA Pro and Ghidra interoperability framework

elias Ghidra, Reverse Engineering

Ghidra has only been released for a short while and the RCE community started adopting it (scripts, tutorials, articles, etc.) really quick. Since Ghidra is free and open-source (coming soon™), I expect a torrent of contributions in the form of tools, plugins and scripts.

I think it is important not to create a schism between IDA users and new Ghidra users because that will not benefit the RCE community. For that reason, I think it would be cool to have a simple way to seamlessly run scripts between the two SRE frameworks.

Today I introduce the Daenerys framework that lets you run IDA scripts in Ghidra and vice versa. Since I am a long term IDA user, it is easier for me to start writing the IDA-to-Ghidra adapters first. As time goes by, I will become more familiar with Ghidra’s APIs and will be able to write the adapters that let users run Ghidra scripts seamlessly in IDA as well.

If you have watched the Game of Thrones show, you are perhaps familiar with Daenerys Targaryen (aka “The mother of dragons”). I chose her name for this project because IDA’s logo is represented by a certain medieval lady and Ghidra by a dragon (some say from the GodZilla lore / King Ghidorah). Having the Daenerys framework’s logo as a lady and a dragon standing on equal footing represents a harmonious relationship between the two.

If you are a graphics designers, I would appreciate it if you can create a logo for the framework better than the one I found using Bing.com’s public domain image search.

The Daenerys framework is still in its infancy and your contributions/suggestions are welcome.

Finally, I hope the RCE community continues to strive and becomes more accessible to everyone interested in the field.

Ghidra: A quick overview for the curious

elias IDA Pro

Ghidra, is a software reverse engineering (SRE) suite of tools developed by NSA’s Research Directorate in support of the Cybersecurity mission. It was released recently and I became curious about it and wanted to check it out.

I have not researched to see if someone else did a similar overview article or not, however, I am writing this article for myself and those who don’t want to run Ghidra themselves and just want to learn a bit about it.

I know that it is unfair to compare Ghidra to IDA Pro, but I cannot help it: I am a long time user of IDA Pro and it is my only point of reference when it comes to reverse engineering tools.

This article is going to be long and will contain lots of screenshots. I just started playing with Ghidra and therefore, I might be wrong or might be presenting inaccurate or incomplete information so please excuse me ahead of time.

Table of contents

Continue reading →

Using Z3 with IDA to simplify arithmetic operations in functions

elias IDAPython, Reverse Engineering, Z3

I have been meaning to learn about SMT based program analysis for a long time and recently I started learning about the topic. There are so many frameworks, articles and tutorials out there that I shall explore as time goes by.

Since I am still learning, I do not claim that the following material is rocket science or advanced by any means, instead, it is very basic and should be approachable enough by absolute beginners to Z3 (a theorem prover from Microsoft Research). All I know so far comes from reading Dennis Yurichev‘s e-book “Quick introduction into SAT/SMT solvers and symbolic execution” in addition to the Z3Py introductory material by Eric (https://github.com/ericpony/z3py-tutorial).

In last week’s blog post, I illustrated how to write a basic emulator to evaluate a function’s return value without running it. In today’s blog post, I am going to show how to convert thousands of arithmetic operations from x86 assembly code into simplified Z3 expressions. Continue reading →

Writing a simple x86 emulator with IDAPython

elias IDAPython, Reverse Engineering , ,

Often times, when I stumble upon IDAPython scripts, I notice that they are using inefficient / incorrect IDAPython APIs to disassemble or decode instructions (for instance using idc.GetMnem() or idc.GetDisasm()). Therefore, in this blog post, I am going to illustrate how to use IDA’s instruction decoding functions from IDAPython in order to write a very simple x86 emulator. The goal is to demonstrate the correct use of instruction decoding IDAPython APIs. By the end of this post, you should be able to solve similar problems using IDAPython. Continue reading →