Hacking with GIT: GIT-Fingerprint Tool Release

This is post number two of four dealing with the aftermath of my BSides Glasgow talk. The slides and a video of the talk are available in the first post:

This one deals with the "new" application fingerprinting check that I discussed. I wrote "new" there because I blogged about it in 2016 and had a PoC tool out there called git-version. As I was going to talk about it at BSides I decided to improve the PoC and make something that should hopefully be more usable.

TOO LONG; DON'T CARE

If you don't care about how it was made, then go straight to get the tool:

https://github.com/secarmalabs/git-fingerprint

The README.md has install instructions and how to launch the command prompt. The suggested workflow in the welcome message shows how to use the script.

To test the script I cloned a copy of "CVE-Offline" (a greppable form of CVE details). I then copied the "cve-offline" folder to "cve-offline-old". I used git "checkout" to rollback the old folder to a previous commit and then I started an HTTP listener to share that folder. From there I launched "git-fingerprint" and configured it using these commands:

The following screenshot demonstrates the output from the fingerprinting action:

The highlighted part shows that the target website was using an outdated version of cve-offline.

That should be enough to get you going.

I recommend the rest of this blog only for the brave souls. Intrepid adventurers looking to learn about the technique, design goals, and get a free tutorial in the Python CMD2 module to boot. Should take about 15-20 minutes to read.

VERSION FINGERPRINTING USING GIT

If the application you are targeting is powered by code which you can download the Git repository for. Then you can fingerprint version information using the specific commit dates of files. To do this you:

1. Clone the git repository down
2. Enumerate the files and folders in that repository to make a wordlist
3. Attempt to download the files from your target site
4. Any files which come down as 200 OK, check if the git repository has the exact same version (talking md5/sha1 identical) in its commit history.
5. Display the list of matches along with the commit date.

The above algorithm is the basis of the technique. It is also true that the same algorithm will work for other version control systems (VCS). It is just that I have made the PoC work for Git with that being the most popular VCS at the moment.

Caveat; Files that this will work for are those which are not altered by the download process. PHP/JSP/ASPX etc will all alter to return HTML code instead of the committed file contents. Good candidates are ".js", ".txt", etc.

The technique is generic and will work without the need to maintain a database of regular expressions to detect minor differences in files. Pre-existing fingerprinting techniques rely on such databases to succeed.

WHY VERSION FINGERPRINT AT ALL?

As a professional pentester (or bug bounty ninja) you need to fingerprint the software versions in use by your target. Armed with an accurate version you can:

• Confirm if the version is supported by the vendor - Unsupported software will not receive security updates and will become increasingly vulnerable to attack over time.
• Check if it is the latest release - If it is supported but outdated then there is a deviation from best practice.
• Enumerate known vulnerabilities - Using resources including the changelog of the vendor, or resources like cvedetails etc.

Some targets will have been hardened to prevent trivial version number leaks via HTTP headers, HTML comments etc. This is good because it limits information to an attacker or so it is said.

GOALS FOR THE UPGRADE

As I was going to talk about it at BSides I decided to improve the PoC and make something that should hopefully be more usable. Key parts of the upgrade included:

• Enumerating a repository automatically to find the files and commit count (instead of several "find" commands).
• Attempting to download those files from a target site (instead of "wget")
• List the dates of the commits made for confirmed files (instead of just saying commit "x" of "y" total number of commits.

Basically, take the parts that were done using bash and do them within the script, and improve the output.

DESIGN CHOICES

During my development time I came across the Python CMD2 module. This enables simple command prompt interfaces to be created and does a lot of heavy lifting. I always enjoyed things like the Social Engineering Toolkit which adopt that input approach. I think that makes it easier for a fully featured framework to grow up around a kernel of functionality, so I was interested in trying that out.

Benefits of CMD2:

1. Rapid development of a command prompt interface
2. Relying on "argparse" for implementing individual commands.
3. Tab complete for local paths and commands was easily attainable.
4. Command history via the arrow keys
5. Almost self documenting help pages for individual commands.
6. Usage flexibility - Yes this is a "command prompt interface" but you can also create scripts that can execute a list of commands, and you can call commands from the command line too.
7. CMD2 was also about to drop support for python 2.x meaning I needed to bite hard and go for my first python 3 application.

Essentially I fell in love with CMD2 during early development and couldn't back out. The final section of this blog is an ode to CMD2. It also explains the structure of "Git-Fingerprint" at the same time. Why just drop a tool and not discuss how it is put together for the curious?

CMD2 CRASH COURSE TUTORIAL

You can install cmd2 via pip. As I am using Python3 you would need to use the command shown below:

To access that module within your ".py" simply import it using these lines:

Then declare your class and pass it cmd2.Cmd:

The multiline string here is part of the self-documentation. This string will be displayed whenever the command line interface is launched.

The "prompt" text will be displayed when your script is waiting for user input.

While I was working on Git-Fingerprint I asked the CMD2 project to support categorisation of commands. This was a ticket they had already been asked for and god love them they implemented it the week I asked :D. Much love to the CMD2 maintainers.

A category will control the output of the "help" command. Built in commands from CMD2 will be displayed under one category and your commands can be shown in logical chunks. The following demonstrates that:

To enable that ensure that "with_category" is imported and define your categorisation text using a variable such as "CMD_CAT_GIT_VERSION" as shown previously.

To define a new command inside your interface use code  like the following:

The "@with_argparser" directive takes an ArgParser object and is used to define the inputs and help pages for a command. This is a standard library for defining how to interact with Python scripts. In an attempt to build a framework I located the argparse definitions for this command in a seperate ".py" file. Smaller files with dedicated tasks is *probably* more maintainable.

Any function prefixed with "do_" are picked up by CMD and become commands. The command name will be the text after that prefix so our command is "set_repo_path".

The "@with_category" directive takes the variable "CMD_CAT_GIT_VERSION" that we declared earlier. This marks the "do_set_repo_path" action as part of Git-Fingerprints commands when the "help" command is executed.

The multiline comment at the top becomes the short description of the "do_set_repo_path" command when the "help" command is executed. While the full output of the argparse definition will be displayed if "help set_repo_path" is executed as shown below:

At the end of the interface class definition I also added these important lines:

The previous code block showed the definition of the "set_repo_path" command. Custom commands do not have OS path completion when the user hits "tab" unless you turn it on. The above command is all that you need to turn on tab completion.

As the "set_repo_path" command needs to specify an OS path tab completion was essential.

The above has shown how to:

• define a CMD2 interface
• set custom prompt text
• create a command category and categorise one action
• implement one action called "set_repo_path".
• enable tab completion of paths for "set_repo_path"

The final glue is how to start the interface. At the bottom of the script outside of the "Interface" indenting add this code:

This creates a new instance of the "Interface" class and then starts the CMD2 command loop.

For completeness here is the full listing (you can get the "set_repo_path" file from the Git-Fingerprint repositorty and place it in the same directory if you want to see this work):

Hopefully this has explained the inner workings and means you can go ahead and fix my coding and submit the required updates for me :D

Happy Git Fingerprinting!