How to run a single-threaded tool on multi threads for speed?

If you do this, you are not alone

Introduction

Hey there folks!

Yesterday, I was writing an automating script for something, and it needed me to run a tool on several arguments one at a time. Thing is, the tool was single threaded, so there was no way to, say, just hand the tool a wordlist and have it crunch through it.

Now obviously, if I knew how to code in Go, I would have simply implemented the feature. But I didn’t. Plus, this isn’t the first time that I have come across such situations. I wanted a generic solution. So, I wrote this utility tool.

‘Make-My-Threads‘ — A tool that can run anything with multithreading

‘Make-My-Threads’ is a simple tool that spins up a specified number of concurrent threads and have them execute any specified command; and it supports dynamic arguments (from input-files) as well.

Here’s an example

Let’s say a tool called ‘slow-tool’ needs 2 arguments, one of them is an alphabet, the other is a number, and you invoke it like:

./slow-tool a 1

And you have several argument sets to run (say, a-z, and for each one of them, corresponding 1–9, like ‘a 2’, ‘a 3’,…), but writing a simple bash automation script would still execute the tool slowly, since the commands would run one-by-one. Make-My-Threads can solve this…

The solution

For the above example, you’d probably use Make-My-Threads as:

./make_my_threads.py --threads 50 --command "./slow-tool ARG1 ARG2" --mode clusterbomb -f alphabets.txt:ARG1 -f numbers.txt:ARG2

Where ‘alphabets.txt’ and ‘numbers.txt’ provide the first and second arguments to your chosen command (Prepare these input text files first), and ‘ARG1’ and ‘ARG2’ are placeholders for these arguments. You can use any arbitrary number of dynamic arguments.

Usage

There are 3 modes in which ‘Make-My-Threads’ can work:

• Clusterbomb: Uses every permutation of provided arguments, say 1st-1st,1st-2nd,…
• Pitchfork: Uses only corresponding arguments, say 1st-1st arg, 2nd-2nd,…
• Repeat: Repeats the same static command for specified number of times

Clusterbomb and Pitchfork mode require input files from where to read each set of dynamic command arguments. You can use any arbitrary number of dynamic arguments (and corresponding input-files) Male sure to use proper placeholders that do not occur anywhere else in the command.

Repeat mode only works with static commands, i.e, you cannot use placeholders to have them replaced by inputs from a file.

GitHub Repo

Find ‘Make-My-Threads’ on GitHub here: ‘Make-My-Threads’