apify · daveomri · Jul 30, 2026 · Jun 10, 2026 · Jun 22, 2026 · Jun 22, 2026
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -0,0 +1,65 @@
+# Contributing
+
+Thanks for considering a contribution to `langchain-apify`. This file covers the contribution flow: what to file, how to scope a PR, and what review looks for.
+
+> Setting up locally for the first time? Start with [DEVELOPMENT.md](DEVELOPMENT.md). Install, lint, and test commands live there.
+
+## Filing issues
+
+Open a GitHub issue at <https://github.com/apify/langchain-apify/issues>. A few notes:
+
+- **Bug reports**: include the package version, a short reproducer, the expected behaviour, and what you saw instead. If you have an Apify run ID from the failing call, paste it; it's the fastest way for us to inspect the run server-side.
+- **Feature requests**: describe the use case before the proposed API. We'd rather discuss the shape of the solution before code lands.
+- Search existing issues first; the maintainers may already be tracking the same thing.
+
+## Working on a pull request
+
+- Branch from `main`.
+- Keep one logical change per PR. A PR that fixes a bug *and* adds a feature is harder to review and harder to revert if needed.
+- Before opening: run `make lint` and `make test` locally. CI will also run integration tests; you don't need an `APIFY_TOKEN` to open the PR (CI has its own).
+- If your change affects the public API, update the README and any in-repo examples.
+- If your change adds a new tool family or generic primitive, add or extend the corresponding test file under `tests/unit_tests/`.
+
+## Commit message conventions
+
+The release workflow uses `git-cliff` to read commit-message prefixes and auto-generate both the version bump and the changelog. **Don't manually edit `version =` in `pyproject.toml` or write `CHANGELOG.md` entries by hand.**
+
+Use these prefixes:
+
+- `feat:` for a new feature → minor version bump
+- `fix:` for a bug fix → patch bump
+- `ref:` for a refactor → patch
+- `test:` for a test-only change → patch
+- `chore:` for housekeeping → patch
+- `docs:` for a docs-only change → no bump on a stable release (skipped from pre-releases)
+- `ci:` for a CI / workflow change (skipped from pre-releases)
+
+A `BREAKING CHANGE:` footer in the commit body triggers a major bump. Use it sparingly and only when the public API genuinely breaks.
+
+A good message is short on the subject line and explains *why* (not what) in the body:
+
+```
+fix: forward apify_token from ApifyWrapper to ApifyDatasetLoader
+
+Without this, an explicit wrapper token still required APIFY_TOKEN to
+also be set in the environment. The loader fell back to env-var
+resolution and raised ValueError if neither was present.
+```
+
+## What review looks for
+
+- **Correctness on the public API surface.** Any new tool must follow the `_ApifyGenericTool` envelope contract (a JSON string of `{"run": {...}, "items": [...]}`) and route Actor calls through `ApifyToolsClient` (`_client.py`), not the SDK directly.
+- **`make lint` and `make test` pass locally.** Integration tests pass under CI's token; you don't need to run them yourself unless you're touching `_client.py`.
+- **No new `apify_api_token` field declarations.** The canonical token kwarg is `apify_token`; the legacy `apify_api_token` is honoured only via the existing deprecation plumbing in `_utils.py` and per-tool model validators. New code should not introduce fresh `apify_api_token` fields.
+- **No manually bumped `version =` in `pyproject.toml`.** Versions come from commit messages via `git-cliff`.
+- **Shared defaults stay in `_constants.py`.** Don't reintroduce magic literals (`300`, `100`, `120`, etc.); import the named constant instead.
+
+## Releases
+
+Releases are automated. After a PR merges to `main`, the release workflow:
+
+1. Reads commit messages since the last tag.
+2. Computes the new version with `git-cliff`.
+3. Bumps `pyproject.toml`, writes the changelog, tags the release, and pushes to PyPI.
+
+You don't need to do any of those steps manually.
diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md
@@ -1,8 +1,8 @@
 # Development
 
-## Contributing
+This file covers everything you need to run the code locally: install, format, lint, and test.
 
-If you want to contribute, please ensure that you have your environment properly set up. Run the following commands to make sure that the code is properly formatted and is not breaking before submitting a pull request.
+> Planning a pull request? Read [CONTRIBUTING.md](CONTRIBUTING.md) for issue and PR conventions before opening one.
 
 ## Installation
 
@@ -42,12 +42,14 @@ make test
 To run integration tests, use the following command:
 
 ```bash
-APIFY_API_TOKEN="YOUR_TOKEN" make integration_test
+APIFY_TOKEN="YOUR_TOKEN" make integration_test
 ```
 
 To run single test file, use `TEST_FILE` argument:
 
 ```bash
 make test TEST_FILE=path_to/test_file.py
-APIFY_API_TOKEN="YOUR_TOKEN" make integration_test TEST_FILE=path_to/test_file.py
+APIFY_TOKEN="YOUR_TOKEN" make integration_test TEST_FILE=path_to/test_file.py
 ```
+
+> `APIFY_API_TOKEN` is also accepted as a deprecated alias for `APIFY_TOKEN` (emits a `DeprecationWarning`). New code and examples should use `APIFY_TOKEN`.
diff --git a/README.md b/README.md
@@ -25,7 +25,7 @@ LangChain Apify: A full-stack scraping platform built on Apify's infrastructure
 
 ---
 
-Build web scraping and automation workflows in Python by connecting Apify Actors with LangChain. This package gives you programmatic access to Apify's infrastructure - run scraping tasks, handle datasets, and use the API directly through LangChain's tools.
+Build web scraping and automation workflows in Python by connecting Apify Actors with LangChain. This package gives you programmatic access to Apify's infrastructure: run scraping tasks, handle datasets, and use the API directly through LangChain's tools.
 
 ## Agentic LLMs
 
@@ -39,97 +39,202 @@ pip install langchain-apify
 
 ## Prerequisites
 
-You should configure credentials by setting the following environment variables:
-- `APIFY_API_TOKEN` - Apify API token
+You should configure credentials by setting the following environment variable:
+- `APIFY_TOKEN`: Apify API token. (`APIFY_API_TOKEN` is also honoured as a deprecated alias for backwards compatibility.)
 
 Register your free Apify account [here](https://console.apify.com/sign-up) and learn how to get your API token in the [Apify documentation](https://docs.apify.com/platform/integrations/api).
 
 ## Tools
 
-`ApifyActorsTool` class provides access to [Apify Actors](https://apify.com/store), which are cloud-based web-scraping and automation programs that you can run without managing any infrastructure. For more detailed information, see the [Apify Actors documentation](https://docs.apify.com/platform/actors).
+The package ships dedicated tools across three families plus a generic "wrap any Actor by ID" tool for everything else. All return a uniform `{"run": {...}, "items": [...]}` JSON envelope (parse with `json.loads`).
 
-`ApifyActorsTool` is useful when you need to run an Apify Actor as a tool in LangChain. You can use the tool to interact with the Actor manually or as part of an agent workflow.
+### Core tools
+
+Generic platform primitives: run any Actor or task and fetch dataset items. Available as the convenience list `APIFY_CORE_TOOLS`:
+
+- `ApifyRunActorTool`: start any Actor, return run metadata
+- `ApifyGetDatasetItemsTool`: fetch items from a dataset by ID
+- `ApifyRunActorAndGetDatasetTool`: run + fetch in one call
+- `ApifyScrapeUrlTool`: single URL to markdown
+- `ApifyRunTaskTool`: run a saved Actor task
+- `ApifyRunTaskAndGetDatasetTool`: task run + fetch in one call
 
-Example usage of `ApifyActorsTool` with the [RAG Web Browser](https://apify.com/apify/rag-web-browser) Actor, which searches for information on the web:
 ```python
-import os
-import json
-from langchain_apify import ApifyActorsTool
+import os, json
+from langchain_apify import ApifyRunActorAndGetDatasetTool
 
-os.environ["OPENAI_API_KEY"] = "YOUR_OPENAI_API_KEY"
-os.environ["APIFY_API_TOKEN"] = "YOUR_APIFY_API_TOKEN"
+os.environ["APIFY_TOKEN"] = "YOUR_APIFY_TOKEN"
 
-browser = ApifyActorsTool('apify/rag-web-browser')
-search_results = browser.invoke(input={
-    "run_input": {"query": "what is Apify Actor?", "maxResults": 3}
+result = ApifyRunActorAndGetDatasetTool().invoke({
+    "actor_id": "apify/python-example",
+    "run_input": {"first_number": 2, "second_number": 3},
 })
+print(json.loads(result))
+```
+
+### Search & crawling tools
+
+Web search, maps, video, e-commerce, and content crawling. Available as `APIFY_SEARCH_TOOLS`:
+
+- `ApifyGoogleSearchTool`: Google search results
+- `ApifyWebCrawlerTool`: multi-page website crawler
+- `ApifyRAGWebBrowserTool`: search + fetch top results in one call
+- `ApifyGoogleMapsTool`: places, reviews, business details
+- `ApifyYouTubeScraperTool`: videos, channels, metadata
+- `ApifyEcommerceScraperTool`: product pages and category listings
+
+```python
+import os, json
+from langchain_apify import ApifyGoogleSearchTool
+
+os.environ["APIFY_TOKEN"] = "YOUR_APIFY_TOKEN"
+
+result = ApifyGoogleSearchTool().invoke({
+    "query": "langchain apify integration",
+    "max_results": 5,
+})
+print(json.loads(result))
+```
+
+### Social media tools
+
+Instagram, LinkedIn, Twitter/X, TikTok, and Facebook. Available as `APIFY_SOCIAL_TOOLS`:
 
-# use the tool with an agent
+- `ApifyInstagramScraperTool`: profiles, hashtags, posts, comments
+- `ApifyLinkedInProfilePostsTool`: posts from a LinkedIn profile
+- `ApifyLinkedInProfileSearchTool`: keyword search for profiles
+- `ApifyLinkedInProfileDetailTool`: full profile detail
+- `ApifyTwitterScraperTool`: tweets and users
+- `ApifyTikTokScraperTool`: videos, users, hashtags
+- `ApifyFacebookPostsScraperTool`: public page posts
+
+```python
+import os, json
+from langchain_apify import ApifyInstagramScraperTool
+
+os.environ["APIFY_TOKEN"] = "YOUR_APIFY_TOKEN"
+
+result = ApifyInstagramScraperTool().invoke({
+    "search_type": "user",
+    "search_query": "apify",
+    "max_results": 3,
+})
+print(json.loads(result))
+```
+
+### Using tools with an agent
+
+Each convenience list lets you bind a whole tool family to an agent in one line. Don't bind all tools at once. Most LLMs lose routing accuracy past ~8 tools, so pick the family the agent actually needs.
+
+```python
+import os
+from langchain_apify import APIFY_SEARCH_TOOLS
 from langchain_openai import ChatOpenAI
 from langgraph.prebuilt import create_react_agent
 
-model = ChatOpenAI(model="gpt-4o-mini")
-tools = [browser]
+os.environ["OPENAI_API_KEY"] = "YOUR_OPENAI_API_KEY"
+os.environ["APIFY_TOKEN"] = "YOUR_APIFY_TOKEN"
+
+model = ChatOpenAI(model="gpt-5.4-mini")
+tools = [tool_cls() for tool_cls in APIFY_SEARCH_TOOLS]
 agent = create_react_agent(model, tools)
 
 for chunk in agent.stream(
-    {"messages": [("human", "search for what is Apify?")]},
-    stream_mode="values"
+    {"messages": [("human", "search the web for what Apify Actors are")]},
+    stream_mode="values",
 ):
     chunk["messages"][-1].pretty_print()
 ```
 
+### `ApifyActorsTool`: wrap any Actor by ID
+
+For Actors without a dedicated wrapper above, `ApifyActorsTool` builds an input schema from the Actor's build at construction time and exposes it as a generic LangChain tool:
+
+```python
+import os
+from langchain_apify import ApifyActorsTool
+
+os.environ["APIFY_TOKEN"] = "YOUR_APIFY_TOKEN"
+
+tool = ApifyActorsTool("apify/rag-web-browser")
+result = tool.invoke(input={
+    "run_input": {"query": "what is an Apify Actor?", "maxResults": 3},
+})
+```
+
+## Retriever
+
+`ApifySearchRetriever` is a `BaseRetriever` over `apify/rag-web-browser` for RAG pipelines. Each result becomes a LangChain `Document` with `metadata['source']`, `metadata['title']`, and any additional fields the Actor returns.
+
+```python
+import os
+from langchain_apify import ApifySearchRetriever
+
+os.environ["APIFY_TOKEN"] = "YOUR_APIFY_TOKEN"
+
+retriever = ApifySearchRetriever(max_results=3)
+docs = retriever.invoke("what is web scraping")
+for doc in docs:
+    print(doc.metadata["source"], "-", doc.metadata.get("title"))
+```
+
 ## Document loaders
 
-> **⚠️ Note for Actor Developers**: If you're building an Apify Actor, use `Actor.open_dataset()` from the Apify SDK instead of this loader. See the [Note for Apify Actor developers](#note-for-apify-actor-developers) section for details.
+> **⚠️ Note for Actor Developers**: If you're building an Apify Actor, use `Actor.open_dataset()` from the Apify SDK instead of these loaders. See the [Note for Apify Actor developers](#note-for-apify-actor-developers) section for details.
 
-`ApifyDatasetLoader` class provides access to [Apify datasets](https://docs.apify.com/platform/storage/dataset) as document loaders. Datasets are storage solutions that store results from web scraping, crawling, or data processing.
+### `ApifyCrawlLoader`
 
-`ApifyDatasetLoader` is useful when you need to process data from an Apify Actor run **from outside the Actor runtime** (e.g., in an external script, notebook, or application). If you are extracting webpage content, you would typically use this loader after running an Apify Actor manually from the [Apify console](https://console.apify.com), where you can access the results stored in the dataset.
+Active crawler that wraps `apify/website-content-crawler`. Crawls a seed URL and returns each page as a `Document` with `metadata = {"source", "title", "crawl_depth"}`. Implements `lazy_load()` for streaming and `load()` for the eager collection.
 
-Example usage for `ApifyDatasetLoader` with a custom dataset mapping function for loading webpage content and source URLs as a list of  `Document` objects containing the page content and source URL.
 ```python
 import os
-from langchain_apify import ApifyDatasetLoader
+from langchain_apify import ApifyCrawlLoader
 
-os.environ["APIFY_API_TOKEN"] = "YOUR_APIFY_API_TOKEN"
+os.environ["APIFY_TOKEN"] = "YOUR_APIFY_TOKEN"
 
-# Example dataset structure
-# [
-#     {
-#         "text": "Example text from the website.",
-#         "url": "http://example.com"
-#     },
-#     ...
-# ]
+loader = ApifyCrawlLoader(
+    url="https://docs.apify.com",
+    max_crawl_pages=5,
+    max_crawl_depth=1,
+)
+documents = loader.load()
+```
+
+### `ApifyDatasetLoader`
+
+Loads an existing Apify dataset by ID and maps items to `Document` objects via a user-supplied function. Useful when you have a dataset from a previous run and want to reshape it for downstream LangChain steps.
+
+```python
+import os
+from langchain_apify import ApifyDatasetLoader
+from langchain_core.documents import Document
+
+os.environ["APIFY_TOKEN"] = "YOUR_APIFY_TOKEN"
 
 loader = ApifyDatasetLoader(
     dataset_id="your-dataset-id",
-    dataset_mapping_function=lambda dataset_item: Document(
-        page_content=dataset_item["text"],
-        metadata={"source": dataset_item["url"]}
+    dataset_mapping_function=lambda item: Document(
+        page_content=item["text"],
+        metadata={"source": item["url"]},
     ),
 )
 ```
 
 ## Wrappers
 
-`ApifyWrapper` class wraps the Apify API to easily convert Apify datasets into documents. It is useful when you need to run an Apify Actor programmatically and process the results in LangChain. Available methods include:
+`ApifyWrapper` is a higher-level facade that runs an Actor (or task) and returns an `ApifyDatasetLoader` over the result dataset. Useful when you want to run an Actor programmatically and process the results in LangChain in a single chain.
 
-- **call_actor**: Runs an Apify Actor and returns an `ApifyDatasetLoader` for the results.
-- **acall_actor**: Asynchronous version of `call_actor`.
-- **call_actor_task**: Runs a saved Actor task and returns an `ApifyDatasetLoader` for the results. Actor tasks allow you to create and reuse multiple configurations of a single Actor for different use cases.
-- **acall_actor_task**: Asynchronous version of `call_actor_task`.
+Methods:
 
-For more information, see the [Apify LangChain integration documentation](https://docs.apify.com/platform/integrations/langchain).
+- `call_actor` / `acall_actor`: run an Actor and return a loader for the results.
+- `call_actor_task` / `acall_actor_task`: run a saved Actor task and return a loader for the results.
 
-Example usage for `call_actor` involves running the [Website Content Crawler](https://apify.com/apify/website-content-crawler) Actor, which extracts content from webpages. The wrapper then returns the results as a list of `Document` objects containing the page content and source URL:
 ```python
 import os
 from langchain_apify import ApifyWrapper
 from langchain_core.documents import Document
 
-os.environ["APIFY_API_TOKEN"] = "YOUR_APIFY_API_TOKEN"
+os.environ["APIFY_TOKEN"] = "YOUR_APIFY_TOKEN"
 
 apify = ApifyWrapper()
 
@@ -138,16 +243,18 @@ loader = apify.call_actor(
     run_input={
         "startUrls": [{"url": "https://python.langchain.com/docs/get_started/introduction"}],
         "maxCrawlPages": 10,
-        "crawlerType": "cheerio"
+        "crawlerType": "cheerio",
     },
     dataset_mapping_function=lambda item: Document(
         page_content=item["text"] or "",
-        metadata={"source": item["url"]}
+        metadata={"source": item["url"]},
     ),
 )
 documents = loader.load()
 ```
 
+For more information, see the [Apify LangChain integration documentation](https://docs.apify.com/platform/integrations/langchain).
+
 ## Note for Apify Actor developers
 
 **If you are building an Apify Actor that will run on the Apify platform**, you should **NOT** use this package for dataset loading. Instead:
@@ -191,3 +298,8 @@ This package is designed for:
 
 It is **NOT** designed for:
 - Code running inside an Apify Actor (use Actor SDK instead)
+
+## Contributing
+
+For local setup (Poetry install, running tests and linting), see [DEVELOPMENT.md](DEVELOPMENT.md).
+For PR scope, commit message conventions, and review expectations, see [CONTRIBUTING.md](CONTRIBUTING.md).
diff --git a/docs/examples/tools_example.py b/docs/examples/tools_example.py
@@ -7,9 +7,9 @@
 from langchain_apify import ApifyActorsTool
 
 os.environ['OPENAI_API_KEY'] = 'YOUR_OPENAI_API_KEY'
-os.environ['APIFY_API_TOKEN'] = 'YOUR_APIFY_API_TOKEN'
+os.environ['APIFY_TOKEN'] = 'YOUR_APIFY_TOKEN'
 
-model = ChatOpenAI(model='gpt-4o-mini')
+model = ChatOpenAI(model='gpt-5.4-mini')
 
 tool = ApifyActorsTool(actor_id='apify/rag-web-browser')