Osintxpress

A fast OSINT data fetching library. Returns Arrow arrays, so you can proceed with your favorite data / visualization tools.

Installation

pip install osintxpress

• arro3-core is the sole dependency, to accept Arrow arrays via the PyCapsule protocol. PyArrow works as well if needed - has more options (such as converting to CSV), but is much bulkier.
• polars is used in the tests.
• Wheels for Win, Mac, Linux, Alpine. FreeBSD wheels can be gladly added once updated arrow / polars packages are available.

Demo

Interactive flight map using Panel and lonboard. osintxpress related code from the demo -

from osintxpress import OsintEngine, SourceAdapter


engine = OsintEngine(worker_threads=2)
engine.add_rest_source(name="flights", adapter=SourceAdapter.OPENSKY, poll_interval_sec=10)
engine.start_all()

.
.

def update_dashboard():

    data = engine.poll()
    if 'flights' in data:
        df = pl.from_arrow(data['flights']).drop_nulls(subset=['longitude', 'latitude'])
    .
    .
    

Run panel serve demo.py, then open http://localhost:5006/demo

Dashboard

osintxpress comes with a complementary dashboard. Used for testing, tweaking etc., it can provide a starting point for Python dashboard builders.

To start the dashboard -

panel serve /path/to/dashboard.py

Supported Sources

osintxpress natively polls / streams from -

Domain	Source	Description
Geopolitics & Conflict	ACLED	Armed Conflict Location & Event Data Project
	UCDP	Uppsala Conflict Data Program (Georeferenced events)
	GDELT	Global Database of Events, Language, and Tone
	OREF	Israel Home Front Command (Real-time rocket/drone alerts)
	UNHCR	Refugee and Internally Displaced Person (IDP) population flows
Cyber & Infrastructure	Cloudflare Radar	Global BGP leaks and route hijacks
	URLhaus	Active malware distribution URLs (abuse.ch)
	Feodo Tracker	Active Botnet Command & Control (C2) servers (abuse.ch)
	Ransomware.live	Recent ransomware group victim postings
	NGA Warnings	Maritime navigational warnings (e.g., Submarine cable repairs)
Natural & Environmental	NASA FIRMS	VIIRS Satellite thermal anomalies (Wildfires & kinetic impacts)
	NASA EONET	Earth Observatory Natural Event Tracker
	USGS	Global live earthquake feeds
	NWS	US National Weather Service active alerts
	Open-Meteo	Live global surface temperatures and weather
Markets & Macro	Polymarket	Decentralized prediction market odds and volume
	FRED	Federal Reserve Economic Data (Macro indices)
	CoinGecko	Real-time cryptocurrency prices
	Binance	Live crypto trade execution streams (WebSocket)
Tracking & Telecom	OpenSky Network	Live global ADS-B flight tracking
	AIS Stream	Live global maritime shipping positions (WebSocket)
	CelesTrak	Active satellite TLE data (Orbital surveillance)
	Telegram	Authenticated MTProto channel scraping via grammers
News & Media	Google News	Accelerated proxy for Reuters World News
	BBC & Al Jazeera	Breaking world news RSS feeds

Suggestions

Feel free to open a ticket or a discussion.