Binary Code Analysis
FIT-IANAcad. year: 2020/2021
This course deepens the knowledge and skill obtained in the course of Machine level programming (ISU) and in the course of Operating systems (IOS) with the main goal of allowing the students to understand the image of a crashed system (the so-called crash dump). Knowledge of programming on the level of assembler and application binary interfaces (ABI) is applied on a real Unix operating system. Within the course, various binary files used in the system are discussed, including their structure and their disassembled contents. The course involves a detailed study of compiler output from the point of view of linkage and run of system code as well as a discussion of differences and relations among various processor architectures, compilers, and application binary interfaces. Next, the course presents selected concepts typical for kernel-level programming whose deep knowledge is necessary for analysing the functionality of an operating system kernel. These concepts include, among others, details of interrupt processing, task queues, process/thread synchronisation and memory management inside the kernel, i.e., the so-called SLAB allocator. Knowledge obtained in this way is used as a basis for presenting possibilities of monitoring the behaviour of an operating system on the binary level during its run as well as analysis of images of system memory after a system crash (crash dump analysis). In both cases, usage of tools shipped within common Unix distributions is accented.
Learning outcomes of the course unit
Practical experience with analysing the image of system memory after a system crash (crash dump analysis). Knowledge of the structure of binary files used in Unix systems (ELF). Understanding differences and relations between processors architectures, compilers, and ABI standards. Students who successfully pass the course will further be able to monitor the run of an operating system on a binary level during its runtime too.
Improved knowledge in the areas of operating systems, machine languages, and debugging and analysis.
Recommended optional programme components
Recommended or required reading
Intel Corporation: Intel 64 and IA-32 Architectures Software Developer Manuals, 2015.
Matz, M., Hubicka, J., Mitchell, M.: System V Application Binary Interface, AMD64 Architecture Processor Supplement, 2013.
Ljubuncic, I.: Linux Kernel Crash Book, 2011.
Drake, C., Brown, K.: Panic! UNIX System Crash Dump Analysis, Prentice Hall, 1995.
Hofmann, F.: The Solaris Operating System on x86 Platforms, Crashdump Analysis, Operating System Internals, 2005.
Planned learning activities and teaching methods
Assesment methods and criteria linked to learning outcomes
Projects will be evaluated based on technical reports with a detailed description of the root cause. The reports must be submitted by their deadlines, late submissions will be evaluated by 0 points.
To successfully pass the course, a student needs to get at least 50 points in total out of which at least 24 points must be obtained from the projects and at least 16 points from the final test.
Language of instruction
The goal is to acquaint students with the operation of modern Unix operating systems on a level close to the binary code and with available tools for observing the behaviour of such systems, including, in particular, their post-mortem analysis.
Specification of controlled education, way of implementation and compensation for absences
The obtained knowledge of students is examined through three projects focused on root causing problems in an operating system from a provided crash dump (3x 20 points) and through a final written test (40 points).
Type of course unit
14 hours, optionally
Teacher / Lecturer
- Computer architectures in general, registers, implicit and explicit stack operations. The x86 and x86_64 computer architectures, common instructions. System V ABI on x86_64 architecture, red zone.
- Compilation, linking, and running code. Examples of compiler optimizations, stack optimizations (leaf and tail calls, function inlining), linker script, understanding of the Unix binary file (ELF, objdump).
- Linux crash dump analysis, DWARF debug symbols, using the crash(8) tool.
- Linux crash dump analysis, understanding kernel Oops, kernel flags, sysrq.
- Kernel process and memory management, task_struct, vmas, SLAB allocator.
- Interrupt processing, postponing work, bottom halves, softirqs and tasklets, work queues.
- Live kernel tracing (SystemTap, ftrace), locks, deadlocks and hangs and their analysis and reconstruction.
Exercise in computer lab
12 hours, compulsory
Teacher / Lecturer
- Decomposition of an ELF binary file, decoding its sections, and code disassembling.
- Using the crash(1) tool on Linux.
- Crash dump analysis of a Linux system on the IA-32 architecture.
- Crash dump analysis of a Linux system on the AMD64 architecture.
- System tracing using SystemTap and ftrace.
- Tracing and analysis of system deadlocks.
13 hours, compulsory
Teacher / Lecturer
- Analysis of a crash dump on the IA-32 architecture.
- Analysis of a crash dump on the AMD64 architecture.
- Monitoring of a running system using SystemTap.