How does the code work?
If you ever wanted to know how the internals of pastepwn work together, this might be a good starting point for you. This wiki page covers all the basic components of pastepwn and explains their usage and how it works.
An analyzer is used to search through the content of a paste for certain indicators, such as credit card details, names, email-addresses, passwords, and so on. Pastepwn provides quite a number of analyzers out of the box. Each analyzer tests for one specific thing.
A basic analyzer is built like the BasicAnalyzer class. Apart from the constructor it contains a single method named def match(self, paste) which gets passed a Paste object.
That method returns True if the analyzer found what it was searching for or False if it did not. It can also return a list of matches.
To perform custom actions based on the found pastes it is possible to create a custom action. Pastepwn provides you with some out-of-the-box actions. You can simply create your own actions by subclassing e.g. the BasicAction class.
Each analyzer needs to implement a method def perform(self, paste, analyzer_name=None, matches=None) which is being called when a paste has been matched by an analyzer. The matches object is a list of strings, that the analyzer matched.
The core of pastepwn has several tasks.
- start all the components and handle them
- container for the queues
A queue to contain all the scraped and normalized pastes from different sources.
A queue to contain tuples with the action, the matched paste and the matched analyzer each. It is being used to collect and execute actions after a paste has been matched.
An event which is set, if an exception occurred. It is used to notify all threads that an exception happened and that they should come to an end soon.
Similar to the exception event, but it is set if the application should be stopped normally.
This class handles all the single Scrapers and starts each of them in a single thread. It receives two arguments when being initialized. The first argument is the paste_queue, where new pastes are pushed to, the second argument is the exception_event.
The ScrapingHandler fills the paste queue managed in the pastepwn core class.
For analyzing the pastes there is a component called PasteDispatcher which dispatches the pastes to the different analyzers. Each analyzer runs in its own thread when a new paste is being dispatched. A thread pool could be implemented here in the future to limit the execution of analyzers and limit the available CPU resources.
The ActionHandler checks for new elements in the action queue and executes the actions. Each action gets executed in the single thread of the ActionHandler.
