Creating an Async and Multi-Threaded Port Scanner

An introduction to creating open source cyber security tools

Source: https://www.fullhost.com/blog/what-is-a-pci-scan/

Introduction

This article is meant to serve as an introduction to multi-threaded and asynchronous coding as well as the creation of open source cyber security tools. It will walk you through the creation of a segmentation scanner used in many compliance tests and its distribution via Pypi.

Why Do Segmentation Scan

Segmentation scanning checks to make sure that a computer in one sub-net cannot reach any computers in another. It should, in principal, quickly scan a list of IPs and corresponding ports to make sure none are accessible. Many cyber security compliance criteria require that network segmentation be implemented. For example PCI DSS requires that systems which store, process or transmit credit card data are isolated from those that do not. Segmentation scanning confirms that a computer on a certain subnet cannot reach others in a foreign subset by attempting to connect to all ports of the corresponding subnet. If no ports are accessible the network can be seen as properly segmented.

Most cyber security practices use tools such as nmap to complete segmentation scanning. However, nmap scans ports one at a time, meaning that it can take a long time to check if all ports are open. Moreover, nmap is usually ran manually with no commonly available scripts used to execute nmap scans for compliance testing. The project I will be describing below is a python3 based port scanner which allows a segmentation scans to be executed from within a script. It attempts to finish these scans faster by using asynchronous and multi-threaded function calls. Moreover, this tool can easily be called from within a python script which allows for the addition of extra features such as automatic execution of scans within a specified time range. This project has been created and released as a python package called segScanner.

Segmentation Scan in Python

In order for us to successfully create this python package we must answer four fundamental questions as defined below:

• What are the Inputs, Outputs?
• How do we solve the speed problem?
• How do we structure the project?
• How do we publish the package?

Inputs and outputs are fairly easy to define. Seeing that we are implementing a variation of the nmap software we can simply use the same inputs and outputs. We will be taking a range of ports (80–443 for example), and a subnet (10.10.11.0/24 for example) and printing a list of open & closed ports to screen.

There are many ways to approach the speed-problem from using a language that runs fast when executed (like Go) to implementing an algorithm that can scan ports with the fewest number of possible steps. I see the biggest speed constraint as the fact that ports are scanned serially and not in an async or multi-threaded fashion. So we will be implementing multi-threading and asynchronous function calls.

In order to package the project successfully code will need to be placed in a folder called src with a sub-folder named after the package we are hoping to publish (in this case segScanner). Because I elected to publish this module as a class that can be imported in other peices of python code, the segScanner sub-directory needs to contain a file called __init__.py which acts as the scanner's constructor.

In order to publish the package to pypi for everyone to install we will need to create two files: pyproject.toml & setup.cfg. These will be used to create our package for distribution.

Operating on the inputs

In order to successfully conduct the segmentation scan we will need three variables: a list of IPs called ipRange, a list of ports called portRange, and a timeout variable for ports the eventual handling of timeout errors. These are initialized in the __init__.py function on line 6. The ipRange variable will be converted into a list on line 11 with the function subnetToIPs. We will need to import the ipaddress library for this to function. The portRange variable will be converted into a list on line 23 with the function splitPortRange. The timeout variable defaults to 3 seconds.

Code Snippet 1: Repairing inputs

Async Scanning

In order to assist with solving the speed problem, each port will be scanned using async function calls. This means we will open subsequent port connections before previous requests have returned a response. Ports will be added to a task_queue and there responses will be awaited. Three asynchronously functions are necessary to accomplish this, as defined below:

1. test_port number to asynchronously connect to a port.
2. scan_ports to add ports to the task_queue, begin the scan, and await the response.
3. scanIP to properly execute the scan_ports function and collect responses.

These functions can all be seen in the following code snippet.

Code Snippet 2: Asyncio port scanning

The asyncio package must be imported for this to function properly.

Multi-Threading

In order to make this function even faster we will not scan IPs from the ipRange list in an iterative fashion. Instead we will open a thread for each IP and finish the scans concurrently. This requires that we import the threading library and can be seen below.

Code Snippet 3: Multi-threading accross the IP range

The Seg-Scaner Package

Creating the Package

Creating the package requires two distinct files: pyproject.toml, which allows the package to be installed locally, and setup.cfg which packages the code for distribution on pypi.

The pyproject file can be made with a few fields that describe the project and it’s dependencies. A minimally viable pyproject file can be seen below.

Code Snippet 4: The pyproject.toml file

A minimally viable setup.cfg, used to package this project can also be seen below.

Code Snippet 4: The setup.cfg file

A Note on Creating the README

Open source packages require a good README file. Although it is tempting to write a simple README without much details, a properly written and SEO optimized file can make a world of difference. Not only does it give your project an air of authority it can make the project easier to find and use. Moreover, developers can more easily contribute code according to project guidelines. A good README can be an excelent starting point for helping find, flag, and solve potential issues & enhancements for your project. Doing this is not too difficult if you know the proper headings to include. I elected to include the following seven sections in my file:

• A one sentence overview
• A one paragraph overview
• A section on additional technical details
• A section on install instructions
• A section on contribution guidelines
• A known issues section
• A donation link

You may also elect to include an FAQ. Sometimes contribution guidelines may be a separate file. A software license found in a separate file is a necessity. I also included a code of conduct in a separate file. The REAMDE I wrote was based off of the video below.

Video 1: A guide on writing a good readme.

Packaging, Uploading & Installing the package

Once you have finished developing your software there are two commands you must run in the projects top level directory, using your terminal. They will package your code and publish it to pypi. The commands are as follows:

python -m build #Builds the pack
python3 -m twine upload dist/* --verbose #Uploads the pack to pypi

To install the package run the following command in your terminal:

pip install segScanner==1.0.0

The whole project can be viewed on github. Please take a moment to give the project a star and open a pull request.

GitHub - AleksaZatezalo/seg-scanner

The package can be found on pypi here:

segScanner