One generally associates the terms “buffer overflow (attack)” and “format string attack” with a kind of primary risk. Both represent the threat of an outside attack on applications with an (IP) interface. Applications could be vulnerable and exploitable, in that “sophisticated” input is cleverly passed on to them, such as strings that are too long, strings with embedded control characters, program code, or the like. Both types of attack intend to overwrite memory areas through tricky specifications and passing parameters. Usually, the attacker’s goal is to overwrite the memory with executable machine code to reach high authorization levels within the targeted application. A possible scenario can be the misuse of the web server and its high authorization through attacks on a web application.
A technical basis for attacks is the application of a so-called “stack” as both a temporary and transfer memory through the runtime environment of an application. This program stack is not to be mistaken for the TCP/IP stack. In general, the stack is used to store parameters and save the so-called return address in the context of a subprogram call. Before program X calls subroutine Y and branches into it, the arguments required by routine Y will be “pushed” onto the stack, from which routine Y then “pops” them. Correspondingly, by calling Y from X, the return address – from Y back to X – will also be pushed onto the stack. The nesting of routine calls then lets the stack grow and shrink during runtime – it “swings”. There is one thing to remember. While the Intel stack grows in the direction of a smaller address –from top to bottom–, the z stack, as explained below, grows in the direction of bigger addresses.
The stack can already be implemented either on a processor level through its design and corresponding instructions, or through software emulation. For instance, the stack concept of Intel’s processors is an elementary part of the design and is supported on the level of machine code through corresponding instructions (PUSH, POP, etc.). A specific register, known as the stack register, identifies the stack and allows its addressing.
On the Z platform, the stack is virtually emulated in a “merely” software technical manner. The memory of the z architecture is a large linear memory with 32 or 64 bit addressing, whereas the Intel memory is segmented. A C language program, for instance, running on the z platform has a corresponding memory area within this 32 or 64 bit address space, where it places the emulated stack during runtime. A stack pointer, which identifies the current top address of the stack, moves up and down during program execution and is completely analogous to the special stack register of the Intel processor. From the perspective of the actual effect and function for a running program, there is no real difference between both processor worlds. Altogether the “z stack” is emulated through the runtime environment of the compiler. This reveals the potential for additional protection, in which the runtime environment carries out extensive plausibility checks. It is very important that the program code residing in the z stack can be executed since there is only “the one” memory. This aspect is a recent development and presents a marked difference from the Intel world. To protect programs against stack-based attacks, the new Intel processors can prevent the execution of program code residing in the stack (memory). An entire category of attacks was rendered impossible by this new measure, namely those transferring the executable program code as a command argument.
Let us turn now to the principles of both attacks. Both are based on the fact that programs expect parameter data of fixed length or that parameter data do not exceed a specific number of bytes. These programs do not check whether the incoming arguments are too long or too numerous, or whether they contain special format characters (e.g. “%n”) that affect a reference beyond the intended memory area by a special interpretation through the runtime system (e.g. through the print function). Here comes the point of attack, where an intentional passing of “unsuitable” or “special” arguments can cause a distinct saving of information. This can lead either to an unnoticed stack (memory) overflow of the running program, which can cause the application to crash, or an attempt to take over program control. In the latter case, the goal is to overwrite the memory with program code (machine instructions, such as an SVC instruction) and/or manipulate the return address in order to continue in a particular program code.
Important conclusion: In general, the primary deficiencies of buffer overflow and format string attacks come from the application or routine itself, which does not consistently check received input for plausibility and conformity. The culprit is less likely to be the processor and/or the operating system. Strong attention should be paid to code inspections, which can even be partially automated, particularly during software development. In commercial or open source software, the user’s influence is minimized and the software must be applied “as is”.
Which possibilities for further protection of applications, such as web applications, exist on the z/OS platform in addition to the standard measures for securing applications? Aside from the real-time monitoring of applications with SF-Sherlock, there is the possibility of “encapsulating” the application with SF-Sherlock’s “logical trap” concept. Since there is only partial or no influence on an existing application, the application’s external behavior and character must be categorized and shielded. By describing or recording the “normal behavior” of the application, irregularities can be revealed quickly and securely; for instance, when a web server supplying services to customers suddenly accesses the payroll information and no longer only the product information. This is a real indication of a possible attack. The encapsulation of applications is a successfully applied measure of the SF-Sherlock practice. One important point to consider is that the purpose of each attack is to reach higher authorizations for performing subsequent accesses and operations. In the context of this abuse, regardless of the methods of attack, the buffer overflow and format string attack methods are only two possibilities. There are many others.
SF-Sherlock’s real-time monitoring and logical trap concept lets you achieve a higher and more solid protection of your critical applications.