CSCI 343 - Homework 3

My buffer, it doth runneth over

Due by 11:59:59pm, Thursday 06 Oct 2016


In this assignment, you will be implementing a buffer overflow exploit against a program that is susceptible to a stack overflow. You will do so utilizing a version of the flaw hypothesis methodology and keep track of your results in a research notebook.


As with the other assignments, I encourage you to discuss various approaches with your peers. However, try to avoid getting into specifics as you might give away hints that the other students have not yet revealed to themselves.

Assignment Requirements

For this project, I want you to proceed in an experimental fashion including basic experimental design and recording of observations and results.

Flaw Hypothesis Methodology

Flaw Hypothesis Methodology (FHM) is sometimes referred to as Flaw Hypothesis Testing. It is a procedure used in system analysis and penetration testing. The general procedure is as follows:

  1. Identify the target system.
  2. Consult the system specification and any available documentation.
  3. Generate a list of hypothetical (but testable) flaws that might exist in the system.
  4. Construct an experiment to test the validity of a given hypothesis. It is important to identify in advance how you will know if your hypothesis is valid.
  5. Repeat steps 2-5 until successful/satisfied.

Research Notebook

I would like for you to keep a research notebook for this project. In this notebook, you will record the various details that you discover, your brainstormed hypotheses, the description of your experimental design, and the results and conclusions from your experiments.

I am interested in seeing the process through which you eventually construct your successful exploit. This means that I am interested in seeing the ideas that you develop and which ones you select for testing, as well as what experiments that you tried and found to be unsuccessful.

This "notebook" need not be a paper record -- though it can be if you so desire. A simple text file will be sufficient. You will be submitting this via Blackboard, so starting with an electronic format is probably easier. If you are working with a partner, something like a Google doc is probably best.

Vulnerable machine

Each student has been given their own instance of an older model GNU/Linux system that has no buffer overflow protections enabled. These are part of the XenLabs system that we've been developing over the past couple of years. Let me know if you run into any issues.

To access your machine, you will need to ssh into the machine using your Blackboard account name and the password that was sent out via email. Your first time connecting it will ask you to change your password from the default to something of your choice. You will then be logged out and have to log in again.

At this point in time you should see a screen very similar to the following:
XenLabs login screen
Select the current lab and you'll see
XenLabs connection screen
You'll want to connect to machine 0 (the only machine that is listed). It will then give you information about connecting to the server to interact with your machine.
XenLabs vnc information

On the lab machines you can go into the menu and then select Network followed by Remote Desktop Viewer. Alternatively, you can just run vinagre from the command line. You will then see:
Vinagre main desktop

Go to Connect and enter the information given to you from the XenLabs screen:
Vinagre selection
If you are going to connect from off campus, you can use OCCS as your host for a SSH tunnel.

At this point it will ask you for the password for that machine (also listed on the XenLabs info page). Anyone with this password can connect to your virtual desktop, so you shouldn't share it. Once you are connected, you should have a fully working remote Linux desktop.
XenLabs remote desktop

Once you click into the window, your mouse will be trapped until you hit CTRL-ALT as the titlebar states. The user account and password are given in the Blackboard announcement. Also, the screen sometimes redraws in a wonky manner. You might need to select View --> Refresh to clear up some artifacts.

Vinagre seems to have issues where it pixels get left behind during screen updates leading to very staticy looking windows. As an alternative I've installed a different VNC client on OCCS that you can use to access things.

    ssh -X

where 59xx are given to you from the "Connect" menu in XenLabs.

You should be all set if you are using a lab machine or operating from a GNU/Linux platform. For others, you may need software for XWindows.


The vulnerable program is a curses-based game called "freesweep", a clone of the popular windows game minesweeper. You should be able to just run it from the command line once you are connected to your computer.

Give it a play!

Capture the flag

On the machine, there is a file located at /usr/local/games/victory.txt which it is your goal to read. You can't do it now, but the freesweep program is setgid games and probably could if it wanted to...

Project Requirements

I would like for you to design a buffer overflow attack against freesweep. You might want to start by reading Aleph One's Smashing the Stack for Fun and Profit. A copy has been conveniently left in the user account on the virtual machine.

There is a published exploit against this program. I'm asking you to not look it up and use it (that's no fun!); however, if you do so, you need to document such in your notebook and/or writeup.

I would like for you to turn in the exploit you wrote (and accompanying tools) as well as your research notebook. In addition, I would like you to create a README file that contains:


I find that there is a thrill from discovering and solving a puzzle on your own. However, I recognize that you might not agree, and possibly might get stuck on your way to a solution. As such, I've prepared a number of hints that you can optionally view to help you on your way.

Feel free to use the following in any order:

  1. Hint 1 - Information about freesweep

  2. Hint 2 - A statically linked binary

  3. Hint 3 - An unstripped binary (w. debug info)

  4. Hint 4 - The source code

  5. Hint 5 - A hint about what data can be used to cause the overflow.

  6. Hint 6 - Exactly what input source can cause an overflow.

  7. Hint 7 - Size matters

  8. Hint 8 - Range of values

Note: once your exploit works, you might not be able to see what you type. That is because freesweep changed some terminal settings. You should be able to get things back to normal by either using the reset command or trying stty sane to get things like the cursor back.

You also can check to see if you are running as part of the games group by using id -a or groups commands.

Once you have it working, you might decide to try out a return-to-libc attack on the same program. (I've not tested this yet, but it might be fun!)

Last Modified: October 05, 2016 - Benjamin A. KupermanVI Powered