Scweet v5.0

v5.0

Simplified API — cleaner constructor and method names:

• Scweet(cookies_file=...) / Scweet(auth_token=...) / Scweet(cookies=...) replaces Scweet.from_sources(...)
• search() replaces scrape(words=[...])
• get_profile_tweets() replaces timeline via scrape kwargs
• get_user_info() replaces get_user_information()

Simplified config — flat ScweetConfig(concurrency=3, proxy=...) replaces nested ScweetConfig.from_sources(overrides={"pool": {"concurrency": ...}})

Other changes:

• Auto-refresh GraphQL query IDs from X on startup (manifest_scrape_on_init=True)
• Flat package structure (no more v4/ sub-package)
• save_name parameter for custom output file names
• X-Client-Transaction-Id header support (now required by X)
• Updated GraphQL response parsing for X's latest response format
• Removed legacy code paths (legacy_csv resume, scweet.py compat layer, accounts.txt format)
• Rewrote README and DOCUMENTATION.md
• 153 unit tests

pip install -U Scweet

Scweet 4.0.0

Scweet v4 is a major rewrite focused on a cleaner surface and a more reliable core. Tweet search scraping is now API-only (Twitter/X web GraphQL), and Scweet manages local state (accounts, leases/cooldowns, resume checkpoints, manifest cache) in a SQLite DB.

Highlights

• API-only tweet search scraping (GraphQL SearchTimeline) using curl_cffi (sync + async).
• DB-first account provisioning into SQLite from:
  • accounts.txt
  • cookies.json and browser-exported Netscape cookies.txt
  • .env via env_path
  • cookies= payload (cookie dict/list, Cookie header string, auth_token string, JSON string, or file path)
• Unified config via ScweetConfig + Scweet.from_sources(...) (with overrides=... for quick patches).
• Account leasing + limits:
  • Per-account proxy support (stored in DB; set via files or ScweetDB)
  • Daily request/tweet caps + min delay between requests
  • bootstrap_strategy for token vs nodriver cookie bootstrap (auto|token_only|nodriver_only|none)
• Output + resume
  • Output formats: csv|json|both|none
  • Resume modes: legacy_csv, db_cursor, hybrid_safe
  • Optional resume dedupe: output.dedupe_on_resume_by_tweet_id
• Manifest support
  • Default manifest shipped with the package
  • Optional remote refresh on init (update_manifest=True)
• Maintenance helpers
  • ScweetDB helpers (inspect accounts, reset cooldowns/leases, set per-account proxy, opt-in duplicate collapse by auth_token)

Breaking changes (vs v3)

• scrape() / ascrape() now returns list[dict] of raw GraphQL tweet objects (full fidelity).
• CSV output is now a curated, stable “important fields” schema derived from GraphQL.
• Use JSON output (or the returned list) if you need every field.

v3 Compatibility (v4.x)

• Legacy import still works: from Scweet.scweet import Scweet (deprecated; planned removal in v5.0).
• Legacy constructor args and scrape/ascrape signatures preserved.
• Legacy CSV naming and legacy resume semantics preserved when using the legacy facade.

Removed / Cleanups

• Old browser scraping engine codepaths. nodriver remains only for internal login/cookie bootstrap.
• Removed requests fallback for API HTTP (v4 uses curl_cffi).

Known limitations

• Profile/followers/following endpoints are stubbed or incomplete in v4.x.
• Twitter/X behavior can change at any time; account quality and request patterns impact reliability.

Recommended

• For production runs + scaling: use the hosted Apify Actor (free plan/trial available): https://apify.com/altimis/scweet?fpr=a40q9&fp_sid=jeb97
• Library docs: see DOCUMENTATION.md

3.0

• Fully asynchronous architecture for faster, smoother scraping
• No more manual Chromedriver setup – Scweet handles Chromium internally
• Enhanced for personal and research-level scraping
• Follower/following scraping temporarily disabled (to return in future updates)

v1.8

Update README.md

1.6

Update xpath

1.5

Update README.md

1.4

1.4

1.2

Add limit to scrap followers/following and some other fixes.

1.1

Fix env bug.

v0.3.1

new version