Hamburglar – To Accumulate Helpful Data From URLs, Directories, And Files
Written in Python Script
There are 2 variations of hamburglar, full and lite. The principle department is the total model, and hamburglar lite is on a separate department.
Hamburglar
Full fledged scraping software for artifact retrieval from a number of sources. There are some dependencies, so set up them first:
pip3 set up -r necessities.txt
Hamburglar Lite
Multithreaded and recursive listing scraping script. Shops helpful data with the filepath and discovering. Hamburglar lite won’t ever require exterior packages, and can all the time stay as a single script. Setup is so simple as requesting the file and utilizing it:
wget https://uncooked.githubusercontent.com/needmorecowbell/Hamburglar/hamburglar-lite/hamburglar-lite.py
The Hamburglar can discover
- ipv4 addresses (public and native)
- emails
- non-public keys
- urls
- cryptocurrency addresses
- something you may think about utilizing regex filters and yara guidelines
That is designed to be rapidly downloaded and executed on a machine.
Operation
utilization: hamburglar.py [-h] [-g] [-x] [-v] [-w] [-o FILE] [-y YARA] path
positional arguments:
path path to listing, url, or file, relying on flag used
non-obligatory arguments:
-h, –help present this assist message and exit
-g, –git units hamburglar into git mode
-x, –hexdump give hexdump of file
-v, –verbose enhance output verbosity
-w, –web units Hamburgler to net request mode, enter url as path
-o FILE, –out FILE write outcomes to FILE
-y YARA, –yara YARA use yara ruleset for checking
Listing Traversal
- python3 hamburglar.py ~/Listing/
This can recursively scan for recordsdata within the given listing, then analyzes every file for quite a lot of findings utilizing regex filters
Single File Evaluation
- python3 hamburglar.py ~/Listing/file.txt
This can recursively scan for recordsdata within the given listing, then analyzes every file for quite a lot of findings utilizing regex filters
YARA Rule Based mostly Evaluation
- python3 hamburglar.py -y guidelines/ ~/Listing
This can compile the yara rule recordsdata within the guidelines listing after which test them in opposition to each merchandise in Listing.
Git Scraping Mode
- python3 hamburglar.py -g https://www.github.com/needmorecowbell/Hamburglar
Including -y will enable the repo to be scraped utilizing yara guidelines
Net Request Mode
- python3 hamburglar.py -w https://google.com
Including a -w to hamburgler.py tells the script to deal with the trail as a url.
At the moment this doesn’t spider the web page, it simply analyzes the requested html content material
Hex Dump Mode
- python3 hamburglar.py -x ~/file-to-dump
This simply does a hex dump and nothing extra proper now — may very well be piped right into a file
This can ultimately be used for binary evaluation
Ideas
- Including -v will set the script into verbose mode, and -h will present particulars of obtainable arguments
- Including -o FILENAME will set the outcomes filename, that is particularly helpful in scripting conditions the place you may want a number of outcomes tables (ie github repo spidering)
Settings
- whitelistOn: activates or off whitelist checking
- maxWorkers: variety of employee threads to run concurrently when studying file stack
- whitelist: record of recordsdata or directories to solely scan for (if whitelistOn=True)
- blacklist: record of recordsdata, extensions, or directories to dam in scan
- regexList: dictionary of regex filters with filter sort as the important thing
Instance output:
{
“/house/adam/Dev/check/e mail.txt”: {
“emails”: “{‘[email protected]‘}”
},
“/house/adam/Dev/check/email2.txt”: {
“emails”: “{‘[email protected]‘}”
},
“/house/adam/Dev/check/ips.txt”: {
“ipv4”: “{‘10.0.11.2’, ‘192.168.1.1’}”
},
“/house/adam/Dev/check/test2/e mail.txt”: {
“emails”: “{‘[email protected]‘, ‘[email protected]‘}”
},
“/house/adam/Dev/check/test2/ips.txt”: {
“ipv4”: “{‘10.0.11.2’, ‘192.168.1.1’}”
},
“/house/adam/Dev/check/test2/hyperlinks.txt”: {
“website”: “{‘http://login.net.com’}”
}
}
