From 85bcb09336374b52d42e7a3acb7615907700ef6e Mon Sep 17 00:00:00 2001 From: johnnymatthews <9611008+johnnymatthews@users.noreply.github.com> Date: Thu, 20 Aug 2026 11:08:59 +0100 Subject: [PATCH 1/5] Cleans up markdown. --- content/build/build-an-app/index.md | 68 +++++++++---------- .../concepts/views-for-builders/index.md | 4 +- content/build/create-a-view/examples/index.md | 48 ++++++------- content/build/create-a-view/index.md | 28 ++++---- content/build/query-data/index.md | 38 +++++------ 5 files changed, 94 insertions(+), 92 deletions(-) diff --git a/content/build/build-an-app/index.md b/content/build/build-an-app/index.md index 650cc10..7b81ffc 100644 --- a/content/build/build-an-app/index.md +++ b/content/build/build-an-app/index.md @@ -3,52 +3,52 @@ title = "Build an app" aliases = ["/guides", "/guides/building-apps-with-shinzo"] description = "Building dApps with Shinzo App SDK - local-first querying with embedded DefraDB integration" +++ -Building an app with Shinzo is made easy using our [app-sdk](https://github.com/shinzonetwork/app-sdk)! +You can build an app with Shinzo using the [app-sdk](https://github.com/shinzonetwork/app-sdk). -`go get github.com/shinzonetwork/app-sdk` +```shell +go get github.com/shinzonetwork/app-sdk +``` ## Concepts -First, let's review some basic concepts behind working with Shinzo. - -When working with a centralized indexing service, you must choose a from a set of APIs they provide to leverage in your application. The centralized indexing service will work to create complex caching strategies to provide you with responses to your queries (which operate over a very large datset) as quickly as possible. Then, in your application, you'll likely want to create a cache of your own that keeps track of the result of recent queries - that way you can work to minimize latency in your app while also, and importantly, minimizing API usage costs. Shinzo flips this script rather significantly. +When working with a centralized indexing service, you choose from a set of APIs they provide and use them in your application. The centralized indexing service builds complex caching strategies to return responses to your queries (which operate over a very large dataset) as quickly as possible. Then, in your application, you likely create your own cache that tracks the results of recent queries, so you can minimize latency in your app while minimizing API usage costs. Shinzo inverts this model. -With Shinzo, you, the app developer, essentially define the API you want to use. Then, your application client is "pushed" the pre-processed result of that API - this essentially forms a verifiably-correct cache to your application clients. Now, your application can simply make queries against its local cache of the data. No need to maintain a separate cache. No need to re-query an API in order to get the latest data. No need to worry about running webhooks that may end up surprising you with their costs. You simply query the data as frequently as you like. With Shinzo, you don't pay per query, you pay for access to transformed data. +With Shinzo, you, the app developer, define the API you want to use. Your application client is then "pushed" the pre-processed result of that API, which forms a verifiably-correct cache for your application clients. Your application can make queries against its local cache of the data. You don't maintain a separate cache, re-query an API for the latest data, or run webhooks that can surprise you with costs. You query the data as frequently as you like. With Shinzo, you don't pay per query. You pay for access to transformed data. -Shinzo leverages [DefraDB](https://github.com/sourcenetwork/defradb) for a number of purposes. In general, it is expected that apps built using Shinzo will also leverage an embedded instance of Defra in their application. When working with Shinzo, you will create/describe or find a series of View(s). View(s) are collections of pre-processed data needed by applications. Then, your application will be "pushed" the pre-processed data from your View(s). +Shinzo uses [DefraDB](https://github.com/sourcenetwork/defradb) for several purposes. Apps built with Shinzo generally also use an embedded DefraDB instance. When working with Shinzo, you create, describe, or find a series of Views. Views are collections of pre-processed data needed by applications. Your application is then "pushed" the pre-processed data from your Views. ### Example -Let's propose a simple app as an example to illustrate how Shinzo works. This application will simply display a counter for the current number of instances of a specified ERC-20 token, let's say USDC. Let's also say, for arguments sake, that there does not exist a method on the contract where we can query to get the current supply of USDC. Instead, the only way to determine this is to parse through the mint and burn events emitted by the contract. +Consider a simple app to illustrate how Shinzo works. The app displays a counter for the current number of instances of a specified ERC-20 token, such as USDC. For argument's sake, assume the contract has no method to query the current supply of USDC. The only way to determine the supply is to parse the mint and burn events emitted by the contract. -To do this, you would first create a View, describing how to transform primitive data (blocks, logs, transactions, etc.) into a format that works for you. In this case, you would filter logs based on those involving the USDC contract address, decode the logs into events using the contracts ABI, and finally filter for only mint and burn events. The Shinzo Hosts and Generator clients will work together to get you the data you need. Your application client(s) will receive all the mint and burn events on that USDC contract. From here, you can make as many GraphQL queries against those events you've received in order to build your application. Your app client(s) won't receive the underlying primitives (blocks, transactions, logs, etc.), only the filtered and decoded events as described in your View. +To do this, you first create a View that describes how to transform primitive data (blocks, logs, transactions, etc.) into a format you can use. In this case, you filter logs involving the USDC contract address, decode the logs into events using the contract's ABI, and finally filter for only mint and burn events. The Shinzo Hosts and Generator clients work together to deliver the data you need. Your application client(s) receive all the mint and burn events on that USDC contract. From here, you can make as many GraphQL queries against the events you've received to build your application. Your app client(s) won't receive the underlying primitives (blocks, transactions, logs, etc.), only the filtered and decoded events as described in your View. ## Usage -Before using the app-sdk, you'll want to [use Viewkit to create the View(s)](/build/concepts/views-for-builders/) for your app, or build them in the [Shinzo Studio](https://studio.shinzo.network/) browser UI. +Before using the app-sdk, [use Viewkit to create the Views](/build/concepts/views-for-builders/) for your app, or build them in the [Shinzo Studio](https://studio.shinzo.network/) browser UI. Once you've created your Views, the next step is to configure your app. ### Configuration -The app-sdk exposes a variety of configuration options for your application and its embedded Defra instance. While most of these config options will only be useful in some niche cases for power users, some of them are worth calling out directly. +The app-sdk exposes several configuration options for your application and its embedded Defra instance. Most are only useful in niche power-user cases, but a few are worth calling out. -By far the most important configuration variable is `minimum_attestations`: +The most important configuration variable is `minimum_attestations`: ```yaml shinzo: minimum_attestations: 1 ``` -This will set the default minimum attestations required when querying your View(s). Please see the attestations section for more info. +This sets the default minimum attestations required when querying your Views. See the attestations section for more info. ```yaml logger: development: true ``` -This will enable all logs; if excluded, this defaults to false and will silence most of the Defra logs. In general, setting development to false (or omitting it) is highly recommended for production as Defra will produce a lot of logs otherwise. +This enables all logs. If excluded, it defaults to false and silences most of the Defra logs. Setting development to false (or omitting it) is recommended for production, since Defra produces a lot of logs otherwise. -Config can be handled in two different ways. You can simply create the config options by hand - the [app-sdk actually creates a default config](https://github.com/shinzonetwork/app-sdk/blob/main/pkg/defra/defra.go#L23) via this manner that is used in place of a nil config. You can also create a config.yaml file ([example](https://github.com/shinzonetwork/app-sdk/blob/main/config.yaml)) and load it with `config.LoadConfig`. To locate your config.yaml file, you may find `file.FindFile` to be really helpful, especially if working in a test context. e.g. +Config can be handled in two ways. You can create the config options by hand. The [app-sdk creates a default config](https://github.com/shinzonetwork/app-sdk/blob/main/pkg/defra/defra.go#L23) this way, which is used in place of a nil config. You can also create a config.yaml file ([example](https://github.com/shinzonetwork/app-sdk/blob/main/config.yaml)) and load it with `config.LoadConfig`. To locate your config.yaml file, the `file.FindFile` helper is useful, especially in a test context. For example: ```go configPath, err := file.FindFile("config.yaml") @@ -66,7 +66,7 @@ if err != nil { Once you've configured your app, you're ready to start your Defra instance. -First, you'll need to create a `SchemaApplier`. +First, create a `SchemaApplier`. ```go type SchemaApplier interface { @@ -74,14 +74,14 @@ type SchemaApplier interface { } ``` -The app-sdk exposed all the implementations of `SchemaApplier` that we imagine you'll ever need, but you're of course welcome to add any new ones if needed. +The app-sdk exposes all the `SchemaApplier` implementations you'll need, but you can add new ones if needed. ```go type SchemaApplierFromFile struct { DefaultPath string } ``` -Is really useful if you'd like to provide your schema in a file. Again, you may find using `file.FindFile` to be really helpful with this, especially if working in a test context. +This is useful when you want to provide your schema in a file. The `file.FindFile` helper is useful here, especially in a test context. ```go type SchemaApplierFromProvidedSchema struct { @@ -94,7 +94,7 @@ func NewSchemaApplierFromProvidedSchema(schema string) *SchemaApplierFromProvide } } ``` -Is useful if you want to simply provide your schema as a string. +This is useful when you want to provide your schema as a string. Finally, @@ -103,7 +103,7 @@ type MockSchemaApplierThatSucceeds struct{} ``` This is what you'd use if you don't have a schema to apply. -If you're planning to use DefraDB for other use cases besides Shinzo in your application, it is recommended that you provide these other schemas via your `SchemaApplier`. Otherwise, if you're only using Defra for Shinzo, you should use `MockSchemaApplierThatSucceeds`. +If you plan to use DefraDB for other use cases besides Shinzo in your application, provide these other schemas via your `SchemaApplier`. If you're only using Defra for Shinzo, use `MockSchemaApplierThatSucceeds`. ```go myDefraInstance, err := defra.StartDefraInstance(shinzoConfig, &MockSchemaApplierThatSucceeds{}) @@ -112,7 +112,7 @@ if err != nil { } ``` -Don't forget to close your Defra instance when your app exits! +Don't forget to close your Defra instance when your app exits. ```go myDefraInstance.Close(context.Background()) @@ -122,7 +122,7 @@ myDefraInstance.Close(context.Background()) The first step to querying a View is to subscribe to it so that Hosts will begin pushing the View contents to your application client. -You'll need to create View objects for each view. +Create a View object for each view. ```go type View struct { @@ -133,7 +133,7 @@ type View struct { ``` All other fields in the View struct can be ignored. -Then, call `SubscribeTo` on your View(s). +Then, call `SubscribeTo` on your Views. ```go err := myView.SubscribeTo(context.Background(), myDefraInstance) @@ -145,13 +145,13 @@ if err != nil { } } ``` -Note: the "collection already exists" error is common and expected if you have already subscribed to a View. It is for informational purposes and can be safely ignored. Other errors should not be ignored. +Note: the "collection already exists" error is common and expected if you have already subscribed to a View. It is informational and can be safely ignored. Other errors should not be ignored. -This will add the View collection's SDL to your Defra instance (allowing you to query the view) and will add the View as a topic of interest for Defra's passive replication system (communicating to the Hosts that they should send you data for the View). +This adds the View collection's SDL to your Defra instance (allowing you to query the view) and adds the View as a topic of interest for Defra's passive replication system (communicating to the Hosts that they should send you data for the View). -You'll now begin receiving data and can start to query against it. +You now begin receiving data and can query against it. -Query with either `QuerySingle` or `QueryArray` for individual objects or arrays. You'll need to provide a graphql query string and you'll need to define a struct representing the resulting object you hope to receive. +Query with either `QuerySingle` or `QueryArray` for individual objects or arrays. Provide a GraphQL query string and define a struct representing the resulting object you expect to receive. ```go result, err := defra.QuerySingle[MyResultStruct](ctx, myNode, queryString) @@ -161,11 +161,11 @@ results, err := defra.QueryArray[MyResultStruct](ctx, myNode, queryString) ### Attestations -Perhaps one of the most unique features of Shinzo is that it allows you to validate your source info against multiple independent sources; instead of having one Generator who provides all the source primitive data, Shinzo uses multiple and allows you to validate your source data through "attestation records" that are signed off by the various Shinzo Generators that wrote the data. +A distinctive feature of Shinzo is that it lets you validate your source data against multiple independent sources. Instead of one Generator providing all the source primitive data, Shinzo uses multiple Generators and lets you validate your source data through "attestation records" signed off by the various Shinzo Generators that wrote the data. -Using the app-sdk, you can filter out query results that do not meet your specified attestation threshold. For example, if you're dealing with high value transaction(s), you may want to filter out any query results where the underlying data was signed off by less than X Shinzo Generator clients. +Using the app-sdk, you can filter out query results that do not meet your specified attestation threshold. For example, if you're dealing with high-value transactions, you may want to filter out any query results where the underlying data was signed off by fewer than X Shinzo Generator clients. -Attestation Records, like Views, are pre-processed and pushed to your application client. Attestation Records are segmented based on the View (or Primitive) they are attesting to; this means that you can select which Views (or Primitives) you want to receive Attestation Records for. You will not receive Attestation Records for data you aren't interested in. +Attestation Records, like Views, are pre-processed and pushed to your application client. They are segmented based on the View (or Primitive) they attest to, so you can select which Views (or Primitives) you want to receive Attestation Records for. You will not receive Attestation Records for data you aren't interested in. To access Attestation Records for a View (or Primitive), use the `AddAttestationRecordCollection` method. @@ -179,9 +179,9 @@ if err != nil { } } ``` -Note: the "collection already exists" error is common and expected if you have already added Attestation Records for a View (or Primitive). It is for informational purposes and can be safely ignored. Other errors should not be ignored. +Note: the "collection already exists" error is common and expected if you have already added Attestation Records for a View (or Primitive). It is informational and can be safely ignored. Other errors should not be ignored. -This method works very similar to `view.SubscribeTo` - it will add the `AttestationRecord_YourView` collection to your Defra instance's SDL so that it can be queried against and it will add it as a topic for passive replication so that Hosts know to send your app client this data. +This method works like `view.SubscribeTo`. It adds the `AttestationRecord_YourView` collection to your Defra instance's SDL so it can be queried against, and adds it as a topic for passive replication so Hosts know to send your app client this data. The app-sdk can be used to filter out results from queries that do not meet a specified attestation threshold. This can be pre-configured in your config.yaml: @@ -189,8 +189,8 @@ The app-sdk can be used to filter out results from queries that do not meet a sp shinzo: minimum_attestations: 2 ``` -Once configured, you can use `QuerySingleWithConfiguredAttestationFilter` or `QueryArrayWithConfiguredAttestationFilter` (from the `attestation` package) to query objects or arrays respectively. These work similarly to `QuerySingle` and `QueryArray` (from the `defra` package) respectively except they will also filter the results based on that minimum attestation record filter you specified in your config. +Once configured, you can use `QuerySingleWithConfiguredAttestationFilter` or `QueryArrayWithConfiguredAttestationFilter` (from the `attestation` package) to query objects or arrays. These work like `QuerySingle` and `QueryArray` (from the `defra` package), except they also filter the results based on the minimum attestation threshold you specified in your config. -*Please make sure you have added the attestation record (using `AddAttestationRecordCollection`) for whatever collections you query using these methods!* +Note: make sure you have added the attestation record (using `AddAttestationRecordCollection`) for whatever collections you query using these methods. Similarly, you can provide a minimum attestation record threshold as a parameter using `QuerySingleWithAttestationFilter` or `QueryArrayWithAttestationFilter` (from the `attestation` package) for objects or arrays respectively. diff --git a/content/build/concepts/views-for-builders/index.md b/content/build/concepts/views-for-builders/index.md index df5a12e..0f6b1eb 100644 --- a/content/build/concepts/views-for-builders/index.md +++ b/content/build/concepts/views-for-builders/index.md @@ -37,7 +37,7 @@ A View is the fundamental unit produced by Viewkit. Each view is a self-containe Conceptually, a view represents the pipeline: -**indexed primitive data → query → lenses (WASM) → GraphQL schema → consumable API** +indexed primitive data → query → lenses (WASM) → GraphQL schema → consumable API {% mermaid() %} flowchart LR @@ -58,7 +58,7 @@ Generator clients produce six primitive collection types, all prefixed with ` image not found -> library not loaded: libwasmer.dylib +```plaintext +image not found +library not loaded: libwasmer.dylib +``` 1. Move back into the shinzo-view-creator repo if you moved out of it: @@ -79,7 +81,7 @@ Under the hood, it uses `wasmer-go`, which depends on a native dynamic library ( cd shinzo-view-creator ``` -1. Install the Wasmer Go module +1. Install the Wasmer Go module: ```shell go get github.com/wasmerio/wasmer-go@v1.0.4 @@ -90,7 +92,7 @@ Under the hood, it uses `wasmer-go`, which depends on a native dynamic library ( go: added github.com/wasmerio/wasmer-go v1.0.4 ``` - This ensures `wasmer-go` and its packaged native libraries are present in your `GOPATH`. + This makes `wasmer-go` and its packaged native libraries available in your `GOPATH`. ### Environment variables @@ -189,7 +191,7 @@ Now that everything is set up, we can start creating and deploying views. - Updated At: 2026-07-09 09:34:27 +0000 UTC ``` -1. Next we're going to add a query (raw ingest shape). First, define the raw data shape to ingest, e.g. basic EVM logs: +1. Next we're going to add a query (raw ingest shape). First, define the raw data shape to ingest, e.g. raw event logs: ```shell viewkit view add query \ @@ -293,9 +295,9 @@ Now that everything is set up, we can start creating and deploying views. - Updated At: 2026-07-09 09:37:24 +0000 UTC ``` - It now shows both the **query** _and_ the **SDL**. + It now shows both the query and the SDL. -1. Attach a WebAssembly lens that decodes event logs using an ABI. There are the flags we're using: +1. Attach a WebAssembly lens that decodes event logs using an ABI. These are the flags we're using: - `--args`: JSON passed to the lens (here, an ABI definition for the ERC-20 `Transfer` event). - `--label "decode"`: human-readable label for the lens. @@ -339,7 +341,7 @@ Now that everything is set up, we can start creating and deploying views. # - lens "decode" ``` - You should now see the **query**, **SDL**, _and_, the lens `decode`: + You should now see the query, SDL, and the `decode` lens: ```output 📄 View: testdeploy @@ -377,7 +379,7 @@ If you see `libwasmer.dylib` / "image not found" errors, revisit the Wasmer setu You need a wallet to sign deployments to `devnet`. -1. Generate a one: +1. Generate a new one: ```shell viewkit wallet generate @@ -433,7 +435,7 @@ This section is optional, but it's a good idea to check the View within the buil 1. Open the displayed URL in your browser, usually [127.0.0.1:9181](http://127.0.0.1:9181/). 1. You should see a GraphQL Playground. -1. Within thie Playground you can: +1. Within this Playground you can: - Inspect the schema (e.g. see `FilteredAndDecodedLogs`). - Run test queries against your local view. - Verify that your lens is filtering logs as expected. @@ -463,10 +465,10 @@ Once your view behaves correctly locally, you can deploy it to a shared network. ## More examples -For progressively more complex View examples — decoding multiple event types, transaction-based views without lenses, materialized vs on-query views, editing and rolling back views — see the [View examples](/build/create-a-view/examples/) page, which includes both the view definitions and the GraphQL queries you run against them. +For progressively more complex View examples (decoding multiple event types, transaction-based views without lenses, materialized vs on-query views, editing and rolling back views), see the [View examples](/build/create-a-view/examples/) page, which includes both the view definitions and the GraphQL queries you run against them. For the conceptual overview, see [Views for builders](/build/concepts/views-for-builders/). For the full command list, filter operators, VWL wire format, and deploy internals, see the [Viewkit reference](/reference/components/viewkit/). For a deeper dive on lenses, available modules, and how to chain them, see the [Lens reference](/reference/components/lens/). For troubleshooting and common errors, see [Operations: Troubleshooting](/run/operations/troubleshooting/). -## Need Help +## Need help {{ need_help(client="Viewkit", repo_name="shinzo-view-creator", repo="https://github.com/shinzonetwork/shinzo-view-creator/issues") }} diff --git a/content/build/query-data/index.md b/content/build/query-data/index.md index 8341ec7..4aa6562 100644 --- a/content/build/query-data/index.md +++ b/content/build/query-data/index.md @@ -7,10 +7,10 @@ description = "GraphQL query examples and patterns for querying indexed data thr This page lists common GraphQL query examples for indexed chain data. The examples focus on blocks, transactions, attestations, signatures, and document navigation using DocIDs and CIDs. {% admonition(type="note") %} -Collection names are prefixed with `____`, derived from the `chain.name` and `chain.network` settings of the Generator client that indexed the data (for example `____Block` or `Optimism__Mainnet__Block`). The examples below use the `____` placeholder — substitute the prefix that matches your chain. See the [chain config](/run/run-a-generator/config-reference#chain) for details. +Collection names are prefixed with `____`, derived from the `chain.name` and `chain.network` settings of the Generator client that indexed the data (for example `____Block` or `Optimism__Mainnet__Block`). The examples below use the `____` placeholder. Substitute the prefix that matches your chain. See the [chain config](/run/run-a-generator/config-reference#chain) for details. {% end %} -## 1. Querying a Block with Nested Data +## Querying a block with nested data Fetch a single block with nested sub-documents. @@ -62,7 +62,7 @@ Fetch a single block with nested sub-documents. } ``` -## 2. Blocks with Signatures (Verifiability) +## Blocks with signatures (verifiability) Verify who signed a block record and inspect the cryptographic metadata. @@ -83,7 +83,7 @@ Verify who signed a block record and inspect the cryptographic metadata. } ``` -## 3. Fetching a Document by DocID +## Fetching a document by DocID Retrieve an exact document when you already know its `_docID`. @@ -102,11 +102,11 @@ query { } ``` -## 4. Attestations and Document Navigation +## Attestations and document navigation Attestation records link documents to one or more CIDs. These CIDs can then be used to navigate to commit metadata or directly to the underlying document. -### 4.1 AttestationRecord +### AttestationRecord ```graphql { @@ -120,7 +120,7 @@ Attestation records link documents to one or more CIDs. These CIDs can then be u } ``` -**Response** +#### Response ```json [..., { @@ -134,7 +134,7 @@ Attestation records link documents to one or more CIDs. These CIDs can then be u },...] ``` -### 4.2 CID → Commit Details +### CID to commit details Given a CID from an attestation record, you can query commit-level metadata and signatures. @@ -156,7 +156,7 @@ Given a CID from an attestation record, you can query commit-level metadata and } ``` -**Response** +#### Response ```json { @@ -178,7 +178,7 @@ Given a CID from an attestation record, you can query commit-level metadata and } ``` -### 4.3 CID → Document +### CID to document The same CID can be used to directly resolve the document itself. @@ -199,7 +199,7 @@ The same CID can be used to directly resolve the document itself. } ``` -**Response** +#### Response ```json { @@ -221,7 +221,7 @@ The same CID can be used to directly resolve the document itself. } ``` -### 4.4 From CID → Document Directly +### From CID to document directly ```graphql { @@ -240,7 +240,7 @@ The same CID can be used to directly resolve the document itself. } ``` -**Response** +#### Response ```json { @@ -262,7 +262,7 @@ The same CID can be used to directly resolve the document itself. } ``` -## 5. DocID-Based Queries +## DocID-based queries ```graphql { @@ -281,7 +281,7 @@ The same CID can be used to directly resolve the document itself. } ``` -**Response** +#### Response ```json { @@ -303,9 +303,9 @@ The same CID can be used to directly resolve the document itself. } ``` -## 6. Filters, Ordering & Limits +## Filters ordering and limits -Number of Transactions in a Specific Block +### Number of transactions in a specific block ```graphql query { @@ -329,7 +329,7 @@ query { The total transaction count is `highest transactionIndex + 1`. -## 7. Block with Transaction Count +## Block with transaction count ```graphql query { @@ -342,6 +342,6 @@ query { } ``` -## Need Help +## Need help {{ need_help(client="Host", repo_name="shinzo-host-client", repo="https://github.com/shinzonetwork/shinzo-host-client/issues") }} From e144f322c24b8eacc00eaad4112d874b8815df77 Mon Sep 17 00:00:00 2001 From: johnnymatthews <9611008+johnnymatthews@users.noreply.github.com> Date: Thu, 20 Aug 2026 11:35:44 +0100 Subject: [PATCH 2/5] Tidies up Reference section. --- content/reference/changelog/index.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/content/reference/changelog/index.md b/content/reference/changelog/index.md index 37b2d40..cb8b88c 100644 --- a/content/reference/changelog/index.md +++ b/content/reference/changelog/index.md @@ -14,13 +14,13 @@ Track what's new across the Shinzo developer platform: network releases, SDK cha ### Upgrade -**Generator Client** +#### Generator client ```shell docker pull ghcr.io/shinzonetwork/shinzo-generator-client:v0.6.5.3-ethereum-mainnet ``` -**Host Client** +#### Host client ```shell docker pull ghcr.io/shinzonetwork/shinzo-host-client:v0.6.5.3-ethereum-mainnet @@ -39,13 +39,13 @@ docker pull ghcr.io/shinzonetwork/shinzo-host-client:v0.6.5.3-ethereum-mainnet ### Upgrade -**Generator Client** +#### Generator client ```shell docker pull ghcr.io/shinzonetwork/shinzo-generator-client:v0.6.5.2-ethereum-mainnet ``` -**Host Client** +#### Host client ```shell docker pull ghcr.io/shinzonetwork/shinzo-host-client:v0.6.5.2-ethereum-mainnet @@ -68,13 +68,13 @@ docker pull ghcr.io/shinzonetwork/shinzo-host-client:v0.6.5.2-ethereum-mainnet ### Upgrade -**Generator Client** +#### Generator client ```shell docker pull ghcr.io/shinzonetwork/shinzo-generator-client:ethereum-mainnet-latest ``` -**Host Client** +#### Host client ```shell docker pull ghcr.io/shinzonetwork/shinzo-host-client:v0.6.5-ethereum-mainnet From bf10078b2c6aebb2d8f1f6f8a72b3ada168acd87 Mon Sep 17 00:00:00 2001 From: johnnymatthews <9611008+johnnymatthews@users.noreply.github.com> Date: Thu, 20 Aug 2026 12:01:29 +0100 Subject: [PATCH 3/5] Bit of a cleanup across this whole section. --- content/run/get-started/index.md | 20 +++--- .../run/operations/troubleshooting/index.md | 30 +++++---- .../run-a-generator/config-reference/index.md | 10 +-- .../archival-vs-pruned/index.md | 2 +- .../high-throughput-tuning/index.md | 6 +- .../managed-gcp-node/index.md | 4 +- .../nginx-with-snapshots/index.md | 8 +-- .../validator-with-geth/index.md | 4 +- .../guides/quicknode-setup/index.md | 18 +++-- .../hardware-requirements/index.md | 6 +- content/run/run-a-generator/install/index.md | 12 ++-- content/run/run-a-generator/register/index.md | 4 +- .../run/run-a-host/config-reference/index.md | 14 ++-- .../configure-event-filters/index.md | 2 +- .../event-filter-allowlist/index.md | 4 +- .../gcp-local-ssd-raid0/index.md | 4 +- .../prod-vm-nginx-tls/index.md | 6 +- .../snapshot-bootstrap/index.md | 22 +++---- .../watchtower-auto-deploy/index.md | 4 +- .../run-a-host/hardware-requirements/index.md | 8 +-- content/run/run-a-host/install/index.md | 8 +-- content/run/run-a-host/private-hosts/index.md | 6 +- content/run/run-a-host/quickstart/index.md | 66 +++++++++---------- content/run/run-a-host/register/index.md | 14 ++-- 24 files changed, 142 insertions(+), 140 deletions(-) diff --git a/content/run/get-started/index.md b/content/run/get-started/index.md index 3678fc1..4f46646 100644 --- a/content/run/get-started/index.md +++ b/content/run/get-started/index.md @@ -6,14 +6,14 @@ description = "Bring up a Generator client and a Host on one machine, peer them mermaid = true +++ -Run a Shinzo Generator client and a Host client on the same machine, peer them over libp2p, and query indexed chain data through the Host client's GraphQL API. If your node is already reachable, the whole thing takes about ten minutes. +Run a Shinzo Generator client and a Host client on the same machine, peer them over libp2p, and query chain data through the Host client's GraphQL API. If your node is already reachable, the whole thing takes about ten minutes. When you're done you'll have: - A running Generator client pulling blocks from an execution node and signing them. - A running Host client receiving those blocks over P2P and serving them. - A GraphQL query returning real chain data through the Host client. -- A understanding of how the two main Shinzo infrastructure pieces fit together. +- An understanding of how the two main Shinzo infrastructure pieces fit together. ## What you're building @@ -28,7 +28,7 @@ Two containers through one shared Docker bridge. Both containers run on the same ## Prerequisites -- Docker. +- Docker. - Both `curl` and `jq`. - A live execution node exposing JSON-RPC and WebSocket. The Generator client reads from this node; it does not run one for you. Acceptable sources include a node you self-host, a node co-located with a validator, GCP Blockchain Node Engine, or any managed node provider. If your node is behind authentication, see the [Generator client install guide's notes on API keys](/run/run-a-generator/install/#do-you-need-an-api-key). @@ -36,7 +36,7 @@ You don't need a wallet, funds, or a ShinzoHub registration for this quickstart. ## Set your execution node endpoint -Export the URL and (optionally) the API key for your node. The rest of the quickstart references these variables. (The `GETH_*` env var names are historical — the Generator client accepts any compatible JSON-RPC and WebSocket endpoint.) +Export the URL and (optionally) the API key for your node. The rest of the quickstart references these variables. (The `GETH_*` env var names are historical; the Generator client accepts any compatible JSON-RPC and WebSocket endpoint.) ```shell export GETH_RPC_URL="" @@ -54,7 +54,7 @@ The Generator client sits next to a blockchain node, subscribes to new blocks, a | Host port | Container port | What it is | | --- | --- | --- | -| `9181` | `9181` | DefraDB GraphQL API. The query interface for raw indexed data. | +| `9181` | `9181` | DefraDB GraphQL API. The query interface for raw chain data. | | `9171` | `9171` | libp2p P2P port. This is how Hosts subscribe to the Generator client. | | `8080` | `8080` | Health, metrics, and registration endpoints. | @@ -80,17 +80,17 @@ docker run -d \ ghcr.io/shinzonetwork/shinzo-generator-client:ethereum-mainnet-latest ``` -`DEFRADB_KEYRING_SECRET` is the password that protects the Generator client's signing key. The Generator client uses this key to sign every document it produces, which gives downstream consumers a way to verify the data came from a real Generator client. The `testnet-secret` password is fine for this quickstart, but remember use something more secure for anything in a production environment. +`DEFRADB_KEYRING_SECRET` is the password that protects the Generator client's signing key. The Generator client uses this key to sign every document it produces, which gives downstream consumers a way to verify the data came from a real Generator client. The `testnet-secret` password is fine for this quickstart, but remember to use something more secure for anything in a production environment. `DEFRADB_P2P_LISTEN_ADDR` tells DefraDB which interface and port to bind libp2p to inside the container. Binding to `0.0.0.0:9171` means the Host container, running on the same Docker bridge, can reach it. -`INDEXER_START_HEIGHT=0` starts indexing at the current chain tip — no historical backfill. To sync from a specific point instead, set this to that block's height. On a chain with a lot of history, a height far below tip means a lot of data to index, so use a recent block if you just want to confirm the pipeline works. +`INDEXER_START_HEIGHT=0` starts the client at the current chain tip, with no historical backfill. To sync from a specific point instead, set this to that block's height. On a chain with a lot of history, a height far below tip means a lot of data to process, so use a recent block if you just want to confirm the pipeline works. `DEFRADB_PLAYGROUND=true` enables a browser-based GraphQL playground on the API port. ## Read the Generator client's P2P address -The Host client needs two things to connect: +The Host client needs two things to connect: 1. The Generator client's libp2p Peer ID. 1. A multiaddr it can dial. @@ -226,7 +226,7 @@ curl -s http://localhost:8080/health | jq '[.p2p.peers[].id]' ] ``` -If both list each other, libp2p is connected and DefraDB is replicating between them! Data from the Generator client lands in the Host client within a few seconds. +If both list each other, libp2p is connected and DefraDB is replicating between them. Data from the Generator client lands in the Host client within a few seconds. ## Query the Host client @@ -261,7 +261,7 @@ curl -s -X POST http://localhost:9182/api/v0/graphql \ The rows that come back were originally just logs on the source chain, then pulled in by the Generator client over the node's WebSocket, signed, gossiped over libp2p to the Host client, and are now being served back to you over GraphQL. You can also browse this data visually in the [Explorer](https://explorer.shinzo.network/). More queries are on the [Host examples](/build/query-data/) page. {% admonition(type="note") %} -The collection prefix (`Ethereum__Mainnet__` in this example) is derived from the `chain.name` and `chain.network` settings of the image you pulled. If you run a Generator client pointed at a different chain, the prefix changes to match — for example `Optimism__Mainnet__Log`. See the [chain config](/run/run-a-generator/config-reference#chain) for details. +The collection prefix (`Ethereum__Mainnet__` in this example) is derived from the `chain.name` and `chain.network` settings of the image you pulled. If you run a Generator client pointed at a different chain, the prefix changes to match, for example `Optimism__Mainnet__Log`. See the [chain config](/run/run-a-generator/config-reference#chain) for details. {% end %} ## Undo everything diff --git a/content/run/operations/troubleshooting/index.md b/content/run/operations/troubleshooting/index.md index 4bdfead..db3f594 100644 --- a/content/run/operations/troubleshooting/index.md +++ b/content/run/operations/troubleshooting/index.md @@ -32,16 +32,18 @@ The Generator client reads from whatever execution node you point it at. That up - `eth_getUncleByBlockHashAndIndex` - `eth_getBlockReceipts` -> Note: The Generator client only actively calls `eth_getBlockByNumber` and `eth_getTransactionReceipt` to ingest data. The other methods are listed for compatibility. +{% admonition(type="note") %} +The Generator client only actively calls `eth_getBlockByNumber` and `eth_getTransactionReceipt` to ingest data. The other methods are listed for compatibility. +{% end %} ### What happens if I lose my node-identity-key? Can I regenerate it? If you lose your `node-identity-key`, your node's identity is permanently lost. -- The key cannot be regenerated -- You must spin up a new instance of the Generator client -- You must register again with a new identity -- The new node may use the same EVM address, but it will be treated as a new identity +- The key cannot be regenerated. +- You must spin up a new instance of the Generator client. +- You must register again with a new identity. +- The new node may use the same EVM address, but it will be treated as a new identity. To avoid this, always back up your node-identity-key. @@ -53,21 +55,21 @@ By default your DefraDB keys are stored in `~/.defra/keys`. To back them up, sim cp -r ~/.defra/keys /mnt/backup-drive/ ``` -### What type of data is indexed? +### What data does the Generator client read? -All blockchain data is indexed, including blocks, transactions, logs, and storage access lists. The data is indexed by hash (block and transaction), block number, and document. +The Generator client reads all blockchain data, including blocks, transactions, logs, and storage access lists. The data is keyed by hash (block and transaction), block number, and document. ### How much space do I need? -With pruning enabled (the default), the Generator's own data stays bounded at roughly 50 to 100 GB on Ethereum Mainnet; we recommend provisioning 300–500 GB to leave headroom (see [hardware requirements](/run/run-a-generator/hardware-requirements/)). The pruner retains the last 1,000 blocks by default and reclaims older ones. Without pruning, storage grows with chain history. Storage figures differ by chain — see [shinzo.network/chains](https://shinzo.network/chains) for the chains Shinzo supports. +With pruning enabled (the default), the Generator's own data stays bounded at roughly 50 to 100 GB on Ethereum Mainnet; we recommend provisioning 300 to 500 GB to leave headroom (see [hardware requirements](/run/run-a-generator/hardware-requirements/)). The pruner retains the last 1,000 blocks by default and reclaims older ones. Without pruning, storage grows with chain history. Storage figures differ by chain. See [shinzo.network/chains](https://shinzo.network/chains) for the chains Shinzo supports. ### How long does it take to sync? -Sync time depends entirely on the chosen start height. The further back the Generator client begins, the longer it will take to catch up to the current block height. The Generator client processes blocks approximately 2–4 seconds per block. +Sync time depends entirely on the chosen start height. The further back the Generator client begins, the longer it will take to catch up to the current block height. The Generator client processes blocks approximately 2 to 4 seconds per block. ### How do I choose a start height? -The further back you choose, the longer it will take to get to current blocks. However, the further back you index, the more you contribute to the network. +The further back you choose, the longer it will take to get to current blocks. However, the further back you start, the more data you contribute to the network. ### How often is the Generator client updated with new blocks? @@ -75,7 +77,7 @@ The Generator client fetches blocks by block number from the upstream node it is ### How does storage grow over time? -Without pruning, storage grows linearly at roughly 10 GB per 1,000 full blocks on Ethereum Mainnet. Storage growth is not perfectly uniform — early blocks on a chain are often significantly smaller than blocks after major protocol upgrades — and the rate differs between chains. +Without pruning, storage grows linearly at roughly 10 GB per 1,000 full blocks on Ethereum Mainnet. Storage growth is not perfectly uniform. Early blocks on a chain are often significantly smaller than blocks after major protocol upgrades, and the rate differs between chains. With pruning enabled (the default), the pruner removes documents for blocks older than the configured retention window, keeping disk usage bounded. You can also passively prune documents that have already been gossiped, which clears up old blocks and reduces long-term storage pressure. @@ -109,7 +111,7 @@ Generator clients are the write side: they read raw blocks from an execution nod ### Do I need to run my own Generator client to run a Host client? -No, but you do need access to at least one working Generator client. A Host client doesn't read the source chain itself; it receives signed primitives from Generator clients over P2P, so it needs at least one reachable Generator client to sync from. That Generator client doesn't have to be _yours_, you can point at any one you can reach. The image ships with default peers, but they aren't guaranteed to be live, so in practice set `BOOTSTRAP_PEERS` to a Generator client you know is up (see [Install](/run/run-a-host/install/)). +No, but you do need access to at least one working Generator client. A Host client doesn't read the source chain itself; it receives signed primitives from Generator clients over P2P, so it needs at least one reachable Generator client to sync from. That Generator client doesn't have to be _yours_; you can point at any one you can reach. The image ships with default peers, but they aren't guaranteed to be live, so in practice set `BOOTSTRAP_PEERS` to a Generator client you know is up (see [Install](/run/run-a-host/install/)). A public Generator client you can point at is planned; this page will link it once it's live. @@ -184,7 +186,7 @@ See [Install](/run/run-a-host/install/#use-docker) for how to confirm a healthy ## Viewkit -### image not found / library not loaded: libwasmer.dylib +### `image not found / library not loaded: libwasmer.dylib` Viewkit uses the Wasmer runtime to execute WASM lenses locally. If the native library can't be found, any command that touches lenses will fail. @@ -274,7 +276,7 @@ If you need a clean slate, stop the local DefraDB instance (Ctrl+C), delete the ### Is there an `Event` collection? -No. Raw event data lives in the `Log` collection, where `topics` holds indexed parameters and `data` holds non-indexed ones. Use the `decode_log` lens to turn raw logs into decoded, structured output. See the [View examples](/build/create-a-view/examples/) for a complete walkthrough. +No. Raw event data lives in the `Log` collection, where `topics` holds the parameters flagged `indexed` and `data` holds the rest. Use the `decode_log` lens to turn raw logs into decoded, structured output. See the [View examples](/build/create-a-view/examples/) for a complete walkthrough. ### What's the difference between `@materialized(if: true)` and `@materialized(if: false)`? diff --git a/content/run/run-a-generator/config-reference/index.md b/content/run/run-a-generator/config-reference/index.md index d02c746..b1096a9 100644 --- a/content/run/run-a-generator/config-reference/index.md +++ b/content/run/run-a-generator/config-reference/index.md @@ -10,14 +10,14 @@ Env vars override YAML values. Where the shipped `config.yaml` sets a different ## chain -Identifies which EVM chain to index. Collection names are derived as `{name}__{network}__Block`, etc. +Identifies which EVM chain the Generator reads. Collection names are derived as `{name}__{network}__Block`, etc. | Key | Type | Default | Required | Env var | Description | | --- | --- | --- | --- | --- | --- | | `name` | string | `Ethereum` | no | `CHAIN_NAME` | Chain name. Also supports `Arbitrum`, `Optimism`, `Avalanche`, or any EVM chain. | | `network` | string | `Mainnet` | no | `CHAIN_NETWORK` | Network name, for example `Mainnet` or `Testnet`. | -Shinzo supports multiple EVM chains — see [shinzo.network/chains](https://shinzo.network/chains) for the current list. The shipped `config.yaml` lists Arbitrum, Optimism, and Avalanche, and any EVM-compatible chain can likely be indexed by setting `chain.name` and `chain.network` to the correct values and pointing `geth.node_url` at a compatible RPC endpoint. The codebase is being refactored from EVM-only to a `Chain` interface with chain-specific Fetcher and Converter components, which will formalize multi-chain support. See [Chain abstraction](/reference/components/generator-client#chain-abstraction-in-progress) for the current state. +Shinzo supports multiple EVM chains. See [shinzo.network/chains](https://shinzo.network/chains) for the current list. The shipped `config.yaml` lists Arbitrum, Optimism, and Avalanche, and any EVM-compatible chain can likely be read by setting `chain.name` and `chain.network` to the correct values and pointing `geth.node_url` at a compatible RPC endpoint. The codebase is being refactored from EVM-only to a `Chain` interface with chain-specific Fetcher and Converter components, which will formalize multi-chain support. See [Chain abstraction](/reference/components/generator-client#chain-abstraction-in-progress) for the current state. ## defradb @@ -61,7 +61,7 @@ Badger storage engine configuration. All cache and compaction fields map directl ## geth -Connection details for the execution node. The Generator does not run a node. It reads from one you provide. (The `geth` section name is historical — it accepts any compatible JSON-RPC and WebSocket endpoint.) +Connection details for the execution node. The Generator does not run a node. It reads from one you provide. (The `geth` section name is historical; it accepts any compatible JSON-RPC and WebSocket endpoint.) | Key | Type | Default | Required | Env var | Description | | --- | --- | --- | --- | --- | --- | @@ -76,14 +76,14 @@ Controls how the Generator fetches and processes blocks. | Key | Type | Default | Required | Env var | Description | | --- | --- | --- | --- | --- | --- | -| `start_height` | int | 0 | no | `INDEXER_START_HEIGHT` | Block number to start indexing from on first run with no existing data. 0 means auto-detect from chain tip. Must be 0 or higher. | +| `start_height` | int | 0 | no | `INDEXER_START_HEIGHT` | Block number to start reading from on first run with no existing data. 0 means auto-detect from chain tip. Must be 0 or higher. | | `concurrent_blocks` | int | 8 | no | `INDEXER_CONCURRENT_BLOCKS` | Number of blocks to process concurrently. Shipped `config.yaml` sets 1. | | `receipt_workers` | int | 16 | no | `INDEXER_RECEIPT_WORKERS` | Concurrent receipt fetchers per block. Shipped `config.yaml` sets 8. | | `max_docs_per_txn` | int | 1000 | no | `INDEXER_MAX_DOCS_PER_TXN` | Document threshold for single-transaction block creation. Shipped `config.yaml` sets 100. | | `max_tx_docs_per_batch` | int | 0 | no | `INDEXER_MAX_TX_DOCS` | Per-batch document size for transactions. 0 means use `max_docs_per_txn`. Shipped `config.yaml` sets 100. | | `max_log_docs_per_batch` | int | 0 | no | `INDEXER_MAX_LOG_DOCS` | Per-batch document size for logs. 0 means use `max_docs_per_txn`. Shipped `config.yaml` sets 125. | | `max_ale_docs_per_batch` | int | 0 | no | `INDEXER_MAX_ALE_DOCS` | Per-batch document size for access list entries. 0 means use `max_docs_per_txn`. Shipped `config.yaml` sets 500. | -| `blocks_per_minute` | int | 0 | no | `INDEXER_BLOCKS_PER_MINUTE` | Block indexing rate limit. 0 means no limit. Shipped `config.yaml` sets 60. | +| `blocks_per_minute` | int | 0 | no | `INDEXER_BLOCKS_PER_MINUTE` | Block processing rate limit. 0 means no limit. Shipped `config.yaml` sets 60. | | `health_server_port` | int | 8080 | no | `INDEXER_HEALTH_SERVER_PORT` | Health server port. Set to -1 to disable. | | `open_browser_on_start` | bool | false | no | (none) | Auto-open the health page in a browser on startup. | | `start_buffer` | int | 100 | no | `INDEXER_START_BUFFER` | Start this many blocks before chain tip when skipping ahead. | diff --git a/content/run/run-a-generator/deployment-examples/archival-vs-pruned/index.md b/content/run/run-a-generator/deployment-examples/archival-vs-pruned/index.md index 71fb6e4..fe784f4 100644 --- a/content/run/run-a-generator/deployment-examples/archival-vs-pruned/index.md +++ b/content/run/run-a-generator/deployment-examples/archival-vs-pruned/index.md @@ -8,7 +8,7 @@ mermaid = true When to use this: you need to decide whether your Generator keeps all historical data or prunes old blocks. A pruned Generator uses less storage but can still serve snapshots for Host bootstrap. An archival Generator keeps everything but grows without bound. -These scenarios use a supported EVM chain. Shinzo supports multiple EVM chains — see [shinzo.network/chains](https://shinzo.network/chains) for the current list. To target a different chain, change `chain.name` and point the RPC URLs at a compatible node. See the [chain config](/run/run-a-generator/config-reference#chain) for details. +These scenarios use a supported EVM chain. Shinzo supports multiple EVM chains. See [shinzo.network/chains](https://shinzo.network/chains) for the current list. To target a different chain, change `chain.name` and point the RPC URLs at a compatible node. See the [chain config](/run/run-a-generator/config-reference#chain) for details. ## Topology diff --git a/content/run/run-a-generator/deployment-examples/high-throughput-tuning/index.md b/content/run/run-a-generator/deployment-examples/high-throughput-tuning/index.md index f29cf68..efc73a0 100644 --- a/content/run/run-a-generator/deployment-examples/high-throughput-tuning/index.md +++ b/content/run/run-a-generator/deployment-examples/high-throughput-tuning/index.md @@ -8,7 +8,7 @@ mermaid = true When to use this: your Generator needs to catch up from a historical start height quickly, or you want to maximize block processing throughput on a machine with available CPU and memory. -These scenarios use a supported EVM chain. Shinzo supports multiple EVM chains — see [shinzo.network/chains](https://shinzo.network/chains) for the current list. To target a different chain, change `chain.name` and point the RPC URLs at a compatible node. See the [chain config](/run/run-a-generator/config-reference#chain) for details. +These scenarios use a supported EVM chain. Shinzo supports multiple EVM chains. See [shinzo.network/chains](https://shinzo.network/chains) for the current list. To target a different chain, change `chain.name` and point the RPC URLs at a compatible node. See the [chain config](/run/run-a-generator/config-reference#chain) for details. ## Topology @@ -28,7 +28,7 @@ flowchart LR Gen -- "P2P (libp2p)" --> Hosts {% end %} -The Generator fetches blocks from your execution node and processes them with a configurable number of concurrent workers. Receipt fetching happens in parallel per block. Badger cache sizes control how much data stays in memory before hitting disk. The `GOMEMLIMIT` env var tells the Go runtime when to trigger garbage collection, preventing OOM kills under load. +The Generator fetches blocks from your execution node and processes them with a configurable number of concurrent workers. Receipt fetching happens in parallel per block. Badger cache sizes control how much data stays in memory before hitting disk. The `GOMEMLIMIT` env var tells the Go runtime when to trigger garbage collection, which prevents OOM kills under load. ## Prerequisites @@ -140,7 +140,7 @@ services: - `concurrent_blocks: 8`: Process 8 blocks at the same time instead of the shipped default of 1. This is the code default from `applyDefaults`. Increase it if your node can handle parallel requests. See [indexer config](/run/run-a-generator/config-reference#indexer). - `receipt_workers: 32`: Fetch 32 receipts concurrently per block, up from the shipped 8. Receipts are the bottleneck for blocks with many transactions. See [indexer config](/run/run-a-generator/config-reference#indexer). -- `blocks_per_minute: 0`: Disable the rate limit. The shipped `config.yaml` sets 60, which caps indexing speed. Set to 0 for maximum throughput during catch-up. See [indexer config](/run/run-a-generator/config-reference#indexer). +- `blocks_per_minute: 0`: Disable the rate limit. The shipped `config.yaml` sets 60, which caps verifiable indexing speed. Set to 0 for maximum throughput during catch-up. See [indexer config](/run/run-a-generator/config-reference#indexer). - `block_cache_mb: 1024`: Double the shipped 512. More cache means fewer disk reads for recently written blocks. See [defradb store config](/run/run-a-generator/config-reference#defradb-store). - `memtable_mb: 128`: Double the shipped 64. Larger memtables reduce the frequency of flushes to disk. See [defradb store config](/run/run-a-generator/config-reference#defradb-store). - `index_cache_mb: 512`: Double the shipped 256. More index cache speeds up point lookups during pruning and snapshot creation. See [defradb store config](/run/run-a-generator/config-reference#defradb-store). diff --git a/content/run/run-a-generator/deployment-examples/managed-gcp-node/index.md b/content/run/run-a-generator/deployment-examples/managed-gcp-node/index.md index ee1d30a..6b8955d 100644 --- a/content/run/run-a-generator/deployment-examples/managed-gcp-node/index.md +++ b/content/run/run-a-generator/deployment-examples/managed-gcp-node/index.md @@ -8,7 +8,7 @@ mermaid = true When to use this: you want to run the Generator client against GCP Blockchain Node Engine or another managed node provider, authenticating with an API key header. -These scenarios use a supported EVM chain. Shinzo supports multiple EVM chains — see [shinzo.network/chains](https://shinzo.network/chains) for the current list. To target a different chain, change `chain.name` and point the RPC URLs at a compatible node. See the [chain config](/run/run-a-generator/config-reference#chain) for details. +These scenarios use a supported EVM chain. Shinzo supports multiple EVM chains. See [shinzo.network/chains](https://shinzo.network/chains) for the current list. To target a different chain, change `chain.name` and point the RPC URLs at a compatible node. See the [chain config](/run/run-a-generator/config-reference#chain) for details. ## Topology @@ -85,7 +85,7 @@ services: - `GETH_RPC_URL` and `GETH_WS_URL`: Your GCP BNE endpoints. Replace the example URLs with your own BNE instance URLs. See [geth config](/run/run-a-generator/config-reference#geth). - `GETH_API_KEY=`: Your GCP API key. Replace this with the actual key from your GCP console. See [geth config](/run/run-a-generator/config-reference#geth). - `GETH_API_KEY_TYPE=x-goog-api-key`: The header name GCP BNE expects for authentication. See [geth config](/run/run-a-generator/config-reference#geth). -- `INDEXER_START_HEIGHT=0`: Start indexing from the chain tip. See [indexer config](/run/run-a-generator/config-reference#indexer). +- `INDEXER_START_HEIGHT=0`: Start verifiable indexing from the chain tip. See [indexer config](/run/run-a-generator/config-reference#indexer). - `DEFRADB_KEYRING_SECRET=pingpong`: Encryption secret for the DefraDB keyring. Change this to your own secret and keep it consistent across restarts. See [defradb config](/run/run-a-generator/config-reference#defradb). - `GOMEMLIMIT=14GiB`: Go runtime soft memory limit. Set below the container `mem_limit` to leave headroom for non-Go memory. See [env vars](/run/run-a-generator/config-reference#environment-variables). - `SNAPSHOT_ENABLED=false`: Disable snapshots. Enable if you want the Generator to produce snapshot files for Host bootstrap. See [snapshot config](/run/run-a-generator/config-reference#snapshot). diff --git a/content/run/run-a-generator/deployment-examples/nginx-with-snapshots/index.md b/content/run/run-a-generator/deployment-examples/nginx-with-snapshots/index.md index b55e33f..7cae364 100644 --- a/content/run/run-a-generator/deployment-examples/nginx-with-snapshots/index.md +++ b/content/run/run-a-generator/deployment-examples/nginx-with-snapshots/index.md @@ -6,9 +6,9 @@ aliases = ["/generators/deployment-examples/nginx-tls-snapshots"] mermaid = true +++ -When to use this: you want to run a production Generator behind Nginx with TLS, serving snapshot files to Host clients for fast historical bootstraping. +When to use this: you want to run a production Generator behind Nginx with TLS, serving snapshot files to Host clients for fast historical bootstrapping. -These scenarios use a supported EVM chain. Shinzo supports multiple EVM chains — see [shinzo.network/chains](https://shinzo.network/chains) for the current list. To target a different chain, change `chain.name` and point the RPC URLs at a compatible node. See the [chain config](/run/run-a-generator/config-reference#chain) for details. +These scenarios use a supported EVM chain. Shinzo supports multiple EVM chains. See [shinzo.network/chains](https://shinzo.network/chains) for the current list. To target a different chain, change `chain.name` and point the RPC URLs at a compatible node. See the [chain config](/run/run-a-generator/config-reference#chain) for details. ## Topology @@ -100,7 +100,7 @@ services: This Nginx config is drawn from `nginx.conf` (and the copy generated by `indexer-prod-setup.sh`) in the `shinzo-generator-client` repo. It proxies health, registration, registration-app, metrics, snapshot, and schema endpoints, and returns 404 for unmatched routes. The CORS headers allow requests from `shinzo.network` origins. -The repo's `nginx.conf` listens on `8080` as plaintext HTTP (the repo compose maps `8080:8080`) and does not terminate TLS — the mounted certificate files are not referenced. To make this scenario actually terminate TLS as the title implies, the server block below changes `listen 8080;` to `listen 443 ssl;` and adds the `ssl_certificate` / `ssl_certificate_key` directives pointing at the cert and key files mounted from `~/ssl/`. Everything else matches the repo file verbatim: +The repo's `nginx.conf` listens on `8080` as plaintext HTTP (the repo compose maps `8080:8080`) and does not terminate TLS. The mounted certificate files are not referenced. To make this scenario actually terminate TLS as the title implies, the server block below changes `listen 8080;` to `listen 443 ssl;` and adds the `ssl_certificate` / `ssl_certificate_key` directives pointing at the cert and key files mounted from `~/ssl/`. Everything else matches the repo file verbatim: ```nginx events { worker_connections 1024; } @@ -236,7 +236,7 @@ Once the Generator is running, register it with the Shinzo Network. See [Registr ## Gotchas - The original `docker-compose-prod.yml` sets `SNAPSHOT_ENABLED=false`. This scenario changes it to `true` because the purpose is to serve snapshots to Hosts. If you keep it `false`, no snapshot files are produced and the `/snapshots` endpoint returns nothing. -- The Nginx config terminates TLS on port 443 (`listen 443 ssl;` with `ssl_certificate` / `ssl_certificate_key` pointing at `/etc/nginx/ssl/nginx.crt` and `/etc/nginx/ssl/nginx.key`), and the compose maps host `443` to container `443`. You must place your certificate and key at `~/ssl/nginx.crt` and `~/ssl/nginx.key` or Nginx will fail to start. The repo's `nginx.conf` listens on `8080` as plaintext and maps `8080:8080` — it does not terminate TLS — so the `listen … ssl` and `ssl_certificate*` directives above are the only additions this scenario makes beyond the repo file. +- The Nginx config terminates TLS on port 443 (`listen 443 ssl;` with `ssl_certificate` / `ssl_certificate_key` pointing at `/etc/nginx/ssl/nginx.crt` and `/etc/nginx/ssl/nginx.key`), and the compose maps host `443` to container `443`. You must place your certificate and key at `~/ssl/nginx.crt` and `~/ssl/nginx.key` or Nginx will fail to start. The repo's `nginx.conf` listens on `8080` as plaintext and maps `8080:8080` and does not terminate TLS, so the `listen 443 ssl` and `ssl_certificate*` directives above are the only additions this scenario makes beyond the repo file. - The image tag `ghcr.io/shinzonetwork/shinzo-generator-client:standard` in this compose matches `docker-compose-prod.yml`. The repo's `indexer-prod-setup.sh` pins a versioned tag (`ghcr.io/shinzonetwork/shinzo-generator-client:v0.6.5.1-ethereum-mainnet`), and the [install page](/run/run-a-generator/install/) uses `ghcr.io/shinzonetwork/shinzo-generator-client:ethereum-mainnet-latest`. Docker pull/run tag strategy is being consolidated in issues #326 and #327; align with whatever those land on rather than mixing tags across deployments. - `LOG_LEVEL`, `LOG_SOURCE`, and `LOG_STACKTRACE` appear in the original `docker-compose-prod.yml` but are not read by the Generator client, so they are omitted here. `SCHEMA_AUTH_MODE=none` is kept because the Generator client does read it (it controls auth on the `/api/v1/schema` endpoints Nginx proxies). See the [env vars table](/run/run-a-generator/config-reference#environment-variables) for details. - The snapshot directory defaults to `./snapshots` inside the container. Snapshots are written to the DefraDB data directory at `~/shinzo-data/defradb/snapshots` on the host because of the volume mount. Hosts download them through Nginx, not directly from the filesystem. diff --git a/content/run/run-a-generator/deployment-examples/validator-with-geth/index.md b/content/run/run-a-generator/deployment-examples/validator-with-geth/index.md index c9d1532..e935fd8 100644 --- a/content/run/run-a-generator/deployment-examples/validator-with-geth/index.md +++ b/content/run/run-a-generator/deployment-examples/validator-with-geth/index.md @@ -8,7 +8,7 @@ mermaid = true When to use this: you run a validator and want to run the Generator client beside your own Geth node on the same machine or VPC, with no API key and minimal latency. -These scenarios use a supported EVM chain. Shinzo supports multiple EVM chains — see [shinzo.network/chains](https://shinzo.network/chains) for the current list. To target a different chain, change `chain.name` and point the RPC URLs at a compatible node. See the [chain config](/run/run-a-generator/config-reference#chain) for details. +These scenarios use a supported EVM chain. Shinzo supports multiple EVM chains. See [shinzo.network/chains](https://shinzo.network/chains) for the current list. To target a different chain, change `chain.name` and point the RPC URLs at a compatible node. See the [chain config](/run/run-a-generator/config-reference#chain) for details. ## Topology @@ -58,7 +58,7 @@ docker run -d \ - `GETH_RPC_URL=http://localhost:8545`: Geth JSON-RPC on localhost. See [geth config](/run/run-a-generator/config-reference#geth). - `GETH_WS_URL=ws://localhost:8546`: Geth WebSocket on localhost. See [geth config](/run/run-a-generator/config-reference#geth). -- `INDEXER_START_HEIGHT=0`: Start indexing from the chain tip. See [indexer config](/run/run-a-generator/config-reference#indexer). +- `INDEXER_START_HEIGHT=0`: Start verifiable indexing from the chain tip. See [indexer config](/run/run-a-generator/config-reference#indexer). - `DEFRADB_KEYRING_SECRET=testnet-secret`: Encryption secret for the DefraDB keyring. Change this to your own secret and keep it consistent across restarts. See [defradb config](/run/run-a-generator/config-reference#defradb). - `DEFRADB_P2P_ENABLED=true`: Enable P2P networking so the Generator can push data to Hosts. See [defradb p2p config](/run/run-a-generator/config-reference#defradb-p2p). - `DEFRADB_P2P_LISTEN_ADDR=/ip4/0.0.0.0/tcp/9171`: Listen on all interfaces so Hosts outside the machine can connect. See [defradb p2p config](/run/run-a-generator/config-reference#defradb-p2p). diff --git a/content/run/run-a-generator/guides/quicknode-setup/index.md b/content/run/run-a-generator/guides/quicknode-setup/index.md index 67777a4..11fffde 100644 --- a/content/run/run-a-generator/guides/quicknode-setup/index.md +++ b/content/run/run-a-generator/guides/quicknode-setup/index.md @@ -3,12 +3,12 @@ title = "Quicknode setup" description = "A walkthrough for operators who want to run a Shinzo Generator client but would rather pay a managed node provider than run and babysit their own Ethereum execution node." +++ -In this guide, you'll stand up the Shinzo Generator client as a Docker sidecar, point it at a QuickNode HTTPS + WSS endpoint, authenticate with an `x-token` header, and watch it sign and commit blocks from chain tip. We're using Ethereum mainnet in this example; the process is similar or the same for other networks. +In this guide, you'll stand up the Shinzo Generator client as a Docker sidecar, point it at a QuickNode HTTPS + WSS endpoint, authenticate with an `x-token` header, and watch it sign and commit blocks from chain tip. This example uses Ethereum mainnet; the process is similar or the same for other networks. -If you want to participate in the Shinzo by trustlessly reading and signing on-chain data, but you don't want to run your own node then this guide is for you. Most managed node providers, like Quicknode, require a monthly subscription fee, however most offer a free tier for basic testing. This guide also assumed you have a small Linux VM and can run Docker. +If you want to participate in Shinzo by trustlessly reading and signing on-chain data, but you don't want to run your own node, then this guide is for you. Most managed node providers, like QuickNode, require a monthly subscription fee. However, most offer a free tier for basic testing. This guide also assumes you have a small Linux VM and can run Docker. {% admonition(type="note") %} -Installing and running the Generator client does not require you to be a Validator. The separate Registration step, which makes the Generator a recognized source on the Shinzo network, _does_ require an active and bonded validator on your source chain. However, this guide covers install, run, and verify only, and flags registration as an optional next step. +Installing and running the Generator client does not require you to be a validator. The separate Registration step, which makes the Generator a recognized source on the Shinzo network, _does_ require an active and bonded validator on your source chain. However, this guide covers install, run, and verify only, and flags registration as an optional next step. {% end %} ## Prerequisites @@ -16,13 +16,13 @@ Installing and running the Generator client does not require you to be a Validat - A [QuickNode](https://www.quicknode.com/) account. - A Linux VM with: - ~8 GB RAM, ~30 GB free disk. - - **Docker** + the **Docker Compose plugin** installed. + - Docker + the Docker Compose plugin installed. - Port `9171` reachable if you want Hosts to connect over P2P (fine to leave closed for this guide, since the Generator still reads and signs data locally). - - The examples in this guide assume you're running a Debian-based Linux distro, however any distro is fine; you'll just have to tweak some commands to fit your OS. + - The examples in this guide assume you're running a Debian-based Linux distro. However, any distro is fine; you'll just have to tweak some commands to fit your OS. ## Create a QuickNode endpoint -1. In the QuickNode dashboard, create an endpoint. In this guide we're using **Ethereum mainnet** as an example. +1. In the QuickNode dashboard, create an endpoint. This guide uses Ethereum mainnet as an example. 1. Under **Security**, make sure **Token Authentication** is enabled (JWTs can stay disabled). 1. Copy three things from the endpoint's **Connection Details**: - HTTP Provider URL (e.g. `https://alpha-proud-isle.ethereum-mainnet.quiknode.pro/77f6889.../`). @@ -286,7 +286,7 @@ On a healthy start you'll see, in order: 1. Once it reaches the head of the chain, it transitions to waiting for new blocks (HTTP-only polling fallback message, or a short `not available yet, waiting...` line), which is the live-at-tip milestone. -The Generator's default rate limit is 60 blocks/minute, and Ethereum mainnet produces ~5 blocks/minute, so the ~100-block startup gap closes in roughly a couple of minutes. +The Generator's default rate limit is 60 blocks/minute, and Ethereum mainnet produces ~5 blocks/minute, so the ~100-block startup gap closes in a couple of minutes. ## Verify @@ -356,13 +356,11 @@ sudo rm -rf /root/shinzo-data/defradb/* ``` {% admonition(type="warning") %} -Running `rm -rf` is (mostly) irreversable. Data deleted this way is really, _really_ hard to get back. But you already knew that, right? +Running `rm -rf` is irreversible. Data deleted this way cannot be recovered. {% end %} ## Gotchas -Here are a few things that might trip you up. - - **`x-token` isn't `x-api-key`.** QuickNode's RPC endpoints authenticate with the URL token or an `x-token` header. `x-api-key` is only for QuickNode's admin Console API. Pointing `GETH_API_KEY_TYPE` at `x-api-key` against an RPC endpoint will give you `401`s that look mysterious. - **`DEFRADB_KEYRING_SECRET` must be stable.** If it changes between restarts, DefraDB can't load its existing identity and the container will fail to start with "Failed to load existing DefraDB identity." Restore the original value. Generate it once with `openssl rand -hex 32` and treat it like a password. - **Data dir ownership.** The container runs as UID `1001`, but the bind-mounted data dir is often root-owned on first create. If you see "Permission denied on `.defra/keys`", fix it with `chown -R 1001:1001 /root/shinzo-data/defradb`. diff --git a/content/run/run-a-generator/hardware-requirements/index.md b/content/run/run-a-generator/hardware-requirements/index.md index 06bfba2..6f26136 100644 --- a/content/run/run-a-generator/hardware-requirements/index.md +++ b/content/run/run-a-generator/hardware-requirements/index.md @@ -5,9 +5,9 @@ aliases = ["/generator/hardware-requirements", "/generators/hardware-requirement These requirements are for the Generator client itself. It runs as a sidecar next to an execution node (such as Geth), so size the machine for the node first and add the Generator overhead on top. -## Recommended hardware +## Recommended hardware -The Generator client specific hardware requirements depending on which chain the Generator is indexing. +Generator client hardware requirements depend on which chain the Generator reads. ### Ethereum Mainnet @@ -18,7 +18,7 @@ The Generator client specific hardware requirements depending on which chain the | Storage | 300 GB | 500 GB | | Network | 100 Mbps | 1 Gbps | -With pruning enabled (the default), the Generator retains roughly the last 1,000 blocks, so its own data stays bounded at roughly 50 to 100 GB on Ethereum Mainnet. The 300–500 GB figures above are the recommended provisioned disk — the headroom covers growth, snapshot serving, and P2P replication. In archival mode (pruning disabled), storage grows linearly with chain history and on Ethereum Mainnet can exceed 3 TB (see the [FAQ](/run/operations/troubleshooting/) for details on growth rate). Storage growth differs by chain — see [shinzo.network/chains](https://shinzo.network/chains) for the chains Shinzo supports. +With pruning enabled (the default), the Generator retains roughly the last 1,000 blocks, so its own data stays bounded at roughly 50 to 100 GB on Ethereum Mainnet. The 300 to 500 GB figures above are the recommended provisioned disk. The headroom covers growth, snapshot serving, and P2P replication. In archival mode (pruning disabled), storage grows linearly with chain history and on Ethereum Mainnet can exceed 3 TB (see the [FAQ](/run/operations/troubleshooting/) for details on growth rate). Storage growth differs by chain. See [shinzo.network/chains](https://shinzo.network/chains) for the chains Shinzo supports. ## Sizing for your execution node diff --git a/content/run/run-a-generator/install/index.md b/content/run/run-a-generator/install/index.md index 26d8f2f..09f40ed 100644 --- a/content/run/run-a-generator/install/index.md +++ b/content/run/run-a-generator/install/index.md @@ -3,22 +3,22 @@ title = "Install" aliases = ["/generator/install", "/generators/install"] +++ -This page covers installing a Shinzo Generator client with Docker or from source. To complete the generator setup, you must also register it with the Shinzo Network (see [Registration](../register)). +This page covers installing a Shinzo Generator client with Docker or from source. To complete the Generator setup, you must also register it with the Shinzo Network (see [Registration](../register)). -Running the client only requires access to an Ethereum execution node — you do not need to run an Ethereum validator to install or run the Generator. Registration, however, is a separate step that does require you to be an active, bonded validator on your source chain. See the [Registration prerequisites](../register#prerequisites) for details. +Running the client only requires access to an execution node. You do not need to run a validator to install or run the Generator. Registration, however, is a separate step that does require you to be an active, bonded validator on your source chain. See the [Registration prerequisites](../register#prerequisites) for details. ## Hardware recommendations The Generator client is a lightweight sidecar (the binary is approximately 50 MB) that runs next to an execution node. See the [hardware requirements page](../hardware-requirements/) for CPU, RAM, storage, and network sizing, including how to account for the execution node itself. -## Using Docker +## Using Docker These steps use Docker to run the Shinzo Generator client. To build the Generator client from source, see [Building from source](#building-from-source) below. ### Prerequisites - Docker. -- Access to an execution node that exposes JSON-RPC and WebSocket. The Generator client does not run a node for you, it just reads from one. This can be a node you run yourself, a node co-located with your validator, or a managed provider. +- Access to an execution node that exposes JSON-RPC and WebSocket. The Generator client does not run a node for you; it just reads from one. This can be a node you run yourself, a node co-located with your validator, or a managed provider. - A browser wallet setup. This wallet does not need to hold any funds. You do not need to be a validator, or to run a validator, just to install and run the Generator client. Validator requirements only apply to the [Registration](../register) step. @@ -178,7 +178,7 @@ The following ports must be exposed and available on the machine. | Port | Service | | --- | --- | -| `8080` | Health endpoint (`/health`), metrics (`/metrics`), and registration ('/registration'). | +| `8080` | Health endpoint (`/health`), metrics (`/metrics`), and registration (`/registration`). | | `9171` | DefraDB P2P. | | `9181` | DefraDB GraphQL API. | @@ -202,6 +202,6 @@ docker-compose -f ~/docker-compose.yml start The Generator client falls back to HTTP polling. Check that `GETH_WS_URL` is correct and the port is reachable. HTTP-only mode works but is slightly slower. -## Need Help +## Need help {{ need_help(client="Generator", repo_name="shinzo-generator-client", repo="https://github.com/shinzonetwork/shinzo-generator-client/issues") }} diff --git a/content/run/run-a-generator/register/index.md b/content/run/run-a-generator/register/index.md index e777cf4..9b1657b 100644 --- a/content/run/run-a-generator/register/index.md +++ b/content/run/run-a-generator/register/index.md @@ -12,7 +12,7 @@ Running the Generator client only requires an execution node (see [Install](../i Before you start, have the following ready: 1. **An active, bonded chain validator.** The outpost checks that the validator named in your assertion is active and bonded on the source chain. -1. **Your validator's consensus public key.** The key type, format, and lookup tooling are chain-specific — see [Consensus public key](/reference/components/outpost#consensus-public-key) for how to retrieve it on your chain. It is not your withdrawal address or an EVM address. +1. **Your validator's consensus public key.** The key type, format, and lookup tooling are chain-specific. See [Consensus public key](/reference/components/outpost#consensus-public-key) for how to retrieve it on your chain. It is not your withdrawal address or an EVM address. 1. **Access to your validator's withdrawal key.** The assertion is signed with the withdrawal key to prove control of the validator's stake, and the withdrawal address is included in the assertion. See [Validator assertions](/reference/components/outpost#validator-assertions) for the full flow. 1. **A browser wallet** to sign the on-chain registration transaction. @@ -73,6 +73,6 @@ This key defines your node's identity on the network. Back it up so you can rest If this key is lost, and there is no backup available, you will be unable to restore your node with the same identity. {% end %} -## Need Help +## Need help {{ need_help(client="Generator", repo_name="shinzo-generator-client", repo="https://github.com/shinzonetwork/shinzo-generator-client/issues") }} diff --git a/content/run/run-a-host/config-reference/index.md b/content/run/run-a-host/config-reference/index.md index cb28041..eac3bcb 100644 --- a/content/run/run-a-host/config-reference/index.md +++ b/content/run/run-a-host/config-reference/index.md @@ -28,7 +28,7 @@ P2P networking configuration. The Host receives data from Generator clients over | Key | Type | Default | Required | Env var | Description | | --- | --- | --- | --- | --- | --- | | `enabled` | bool | true | no | (none) | Enable P2P networking. Shipped `config.yaml` sets true. | -| `bootstrap_peers` | string array | empty | no | `BOOTSTRAP_PEERS` | P2P bootstrap peer multiaddrs. Env var is comma-separated. Shipped `config.yaml` lists three indexer peers. | +| `bootstrap_peers` | string array | empty | no | `BOOTSTRAP_PEERS` | P2P bootstrap peer multiaddrs. Env var is comma-separated. Shipped `config.yaml` lists three trustless indexer peers. | | `listen_addr` | string | `/ip4/127.0.0.1/tcp/9171` | no | (none) | Multiaddr to listen on for P2P connections. Applied as a fallback in `StartDefraInstance` when empty. Shipped `config.yaml` sets `/ip4/0.0.0.0/tcp/9171`. | | `max_retries` | int | 5 | no | (none) | Connection attempts before marking a peer as failed. Default applied in `network_handler.go`. | | `retry_base_delay_ms` | int | 1000 | no | (none) | Base delay in milliseconds for exponential backoff. Default applied in `network_handler.go`. | @@ -136,12 +136,12 @@ Host-level configuration: lens registry path, health server, and snapshot bootst ### host snapshot -Downloads historical data from an indexer on first startup for fast initial sync. +Downloads historical data from a trustless indexer on first startup for fast initial sync. | Key | Type | Default | Required | Env var | Description | | --- | --- | --- | --- | --- | --- | | `enabled` | bool | false | no | (none) | Enable snapshot bootstrap on startup. Shipped `config.yaml` sets false. | -| `indexer_url` | string | empty | no | (none) | HTTP base URL of the indexer serving snapshots. Shipped `config.yaml` sets `http://35.206.105.60:8080`. | +| `indexer_url` | string | empty | no | (none) | HTTP base URL of the trustless indexer serving snapshots. Shipped `config.yaml` sets `http://35.206.105.60:8080`. | | `historical_ranges` | object array | empty | no | (none) | Block ranges to download during bootstrap. | #### host snapshot historical ranges @@ -153,13 +153,13 @@ Downloads historical data from an indexer on first startup for fast initial sync ## schema -Dynamic schema fetching from an indexer. The Host fetches the indexer's schema on startup so its collection definitions match the Generator's. +Dynamic schema fetching from a trustless indexer. The Host fetches the schema on startup so its collection definitions match the Generator's. | Key | Type | Default | Required | Env var | Description | | --- | --- | --- | --- | --- | --- | -| `indexer_schema_endpoint` | string | `/api/v1/schema` | no | `INDEXER_SCHEMA_ENDPOINT` | HTTP endpoint path on the indexer for fetching the schema. Defaults to `DefaultIndexerSchemaEndpoint` in `config/config.go` when empty. Shipped `config.yaml` sets `/api/v1/schema`. | +| `indexer_schema_endpoint` | string | `/api/v1/schema` | no | `INDEXER_SCHEMA_ENDPOINT` | HTTP endpoint path on the trustless indexer for fetching the schema. Defaults to `DefaultIndexerSchemaEndpoint` in `config/config.go` when empty. Shipped `config.yaml` sets `/api/v1/schema`. | | `http_client_timeout_secs` | int | 30 | no | (none) | HTTP client timeout in seconds for schema fetches. Must be non-negative and cannot exceed 300 (`MaxSchemaHTTPClientTimeout`); 0 defaults to 30 (`DefaultSchemaHTTPClientTimeout`). Shipped `config.yaml` sets 30. | -| `auth_token` | string | empty | yes if the indexer uses `SCHEMA_AUTH_MODE=token` | `INDEXER_SCHEMA_ENDPOINT_AUTH_TOKEN` | Bearer token used to authenticate schema fetches against indexers with `SCHEMA_AUTH_MODE=token` (the Generator's default). Not settable in `config.yaml` (the field uses `yaml:"-"`); provide via `INDEXER_SCHEMA_ENDPOINT_AUTH_TOKEN` or schema fetches receive 401/503 and fall back to the embedded schema. | +| `auth_token` | string | empty | yes if the trustless indexer uses `SCHEMA_AUTH_MODE=token` | `INDEXER_SCHEMA_ENDPOINT_AUTH_TOKEN` | Bearer token used to authenticate schema fetches against trustless indexers with `SCHEMA_AUTH_MODE=token` (the Generator's default). Not settable in `config.yaml` (the field uses `yaml:"-"`); provide via `INDEXER_SCHEMA_ENDPOINT_AUTH_TOKEN` or schema fetches receive 401/503 and fall back to the embedded schema. | ## pruner @@ -176,7 +176,7 @@ Removes old data to keep storage bounded. Defaults are applied by `SetDefaults` ## logger -Logging configuration. The logger is zap-based. +Logging configuration. The logger uses zap. | Key | Type | Default | Required | Env var | Description | | --- | --- | --- | --- | --- | --- | diff --git a/content/run/run-a-host/configure-event-filters/index.md b/content/run/run-a-host/configure-event-filters/index.md index 6b6d511..7c012da 100644 --- a/content/run/run-a-host/configure-event-filters/index.md +++ b/content/run/run-a-host/configure-event-filters/index.md @@ -178,7 +178,7 @@ Key fields: | `logs_processed` | Events captured matching your filter | | `attestations_created` | Data integrity records written | -If `logs_processed` stays at 0 but `blocks_processed` is climbing, either no matching events have occurred in the blocks being indexed, or the connected Generator client is not providing full transaction data (only block headers). If `transactions_processed` is also 0, contact the [Shinzo team](https://discord.shinzo.network/) for access to a full-data Generator. +If `logs_processed` stays at 0 but `blocks_processed` is climbing, either no matching events have occurred in the blocks being scanned, or the connected Generator client is not providing full transaction data (only block headers). If `transactions_processed` is also 0, contact the [Shinzo team](https://discord.shinzo.network/) for access to a full-data Generator. ### Query stored logs diff --git a/content/run/run-a-host/deployment-examples/event-filter-allowlist/index.md b/content/run/run-a-host/deployment-examples/event-filter-allowlist/index.md index e466072..64e0cab 100644 --- a/content/run/run-a-host/deployment-examples/event-filter-allowlist/index.md +++ b/content/run/run-a-host/deployment-examples/event-filter-allowlist/index.md @@ -8,7 +8,7 @@ mermaid = true When to use this: you want your Host to store only specific contract events instead of every document from every Generator. This cuts storage and speeds up view processing. -These scenarios use data from a supported EVM chain. Shinzo supports multiple EVM chains — see [shinzo.network/chains](https://shinzo.network/chains) for the current list. To target a different chain, change the contract addresses and topic hashes to match the target chain. See the [Generator chain config](/run/run-a-generator/config-reference#chain) for details. +These scenarios use data from a supported EVM chain. Shinzo supports multiple EVM chains. See [shinzo.network/chains](https://shinzo.network/chains) for the current list. To target a different chain, change the contract addresses and topic hashes to match the target chain. See the [Generator chain config](/run/run-a-generator/config-reference#chain) for details. ## Topology @@ -183,7 +183,7 @@ For a step-by-step walkthrough of configuring a single USDT Transfer filter, see - `cascade_filters: true` means a `transaction` type filter on a contract address also filters logs and access-list entries from that address. If you want strict per-type matching, set it to false. - The shipped `config.yaml` includes several `shinzo.*` keys that are not in the `config.go` struct and are silently ignored: `wait_for_gaps`, `max_gap_size`, `batch_processing_enabled`, `batch_max_views_per_job`, `batch_query_cache_size`. They have been omitted from this config. See [no-op keys](/run/run-a-host/config-reference#no-op-keys) for the full list. - The `topic0` for ERC-20 Transfer is the same across all tokens because it is derived from the function signature `Transfer(address,address,uint256)`, not from the contract address. To filter a different event, compute its keccak256 hash from the canonical signature. -- The bootstrap peer IDs in this config are the three indexer peers from the shipped `config.yaml` and may be stale. Check the [Shinzo Validators list](https://registration.shinzo.network/validators) for current peers. +- The bootstrap peer IDs in this config are the three trustless indexer peers from the shipped `config.yaml` and may be stale. Check the [Shinzo Validators list](https://registration.shinzo.network/validators) for current peers. ## Need help diff --git a/content/run/run-a-host/deployment-examples/gcp-local-ssd-raid0/index.md b/content/run/run-a-host/deployment-examples/gcp-local-ssd-raid0/index.md index 5d2b008..e80f21b 100644 --- a/content/run/run-a-host/deployment-examples/gcp-local-ssd-raid0/index.md +++ b/content/run/run-a-host/deployment-examples/gcp-local-ssd-raid0/index.md @@ -8,7 +8,7 @@ mermaid = true When to use this: you need high IOPS for DefraDB storage and want to use GCP local SSDs in a RAID-0 array. Local SSDs provide much higher throughput than standard persistent disks but are ephemeral. -These scenarios use data from a supported EVM chain. Shinzo supports multiple EVM chains — see [shinzo.network/chains](https://shinzo.network/chains) for the current list. To target a different chain, change the contract addresses and topic hashes to match the target chain. See the [Generator chain config](/run/run-a-generator/config-reference#chain) for details. +These scenarios use data from a supported chain. Shinzo supports multiple chains. See [shinzo.network/chains](https://shinzo.network/chains) for the current list. To target a different chain, change the contract addresses and topic hashes to match the target chain. See the [Generator chain config](/run/run-a-generator/config-reference#chain) for details. ## Topology @@ -41,7 +41,7 @@ The GCP startup script detects all local SSDs attached to the VM, creates a RAID ## Startup script -This script is drawn from `scripts/gcp-startup-host-local-ssd.sh` in the `shinzo-host-client` repo (two leading comment lines omitted for brevity). It installs Docker, detects local SSDs by their `nvme_card` model, creates a RAID-0 array if there are two or more, formats and mounts the array, then pulls and starts the Host container: +This script is from `scripts/gcp-startup-host-local-ssd.sh` in the `shinzo-host-client` repo (two leading comment lines omitted for brevity). It installs Docker, detects local SSDs by their `nvme_card` model, creates a RAID-0 array if there are two or more, formats and mounts the array, then pulls and starts the Host container: ```shell #!/bin/bash diff --git a/content/run/run-a-host/deployment-examples/prod-vm-nginx-tls/index.md b/content/run/run-a-host/deployment-examples/prod-vm-nginx-tls/index.md index 2890007..ea0992c 100644 --- a/content/run/run-a-host/deployment-examples/prod-vm-nginx-tls/index.md +++ b/content/run/run-a-host/deployment-examples/prod-vm-nginx-tls/index.md @@ -8,7 +8,7 @@ mermaid = true When to use this: you want to run a production Host on a VM with nginx as a reverse proxy, TLS termination, and persistent volumes for DefraDB data, keys, and lens files. -These scenarios use data from a supported EVM chain. Shinzo supports multiple EVM chains — see [shinzo.network/chains](https://shinzo.network/chains) for the current list. To target a different chain, change the contract addresses and topic hashes to match the target chain. See the [Generator chain config](/run/run-a-generator/config-reference#chain) for details. +These scenarios use data from a supported blockchain. Shinzo supports multiple blockchains. See [shinzo.network/chains](https://shinzo.network/chains) for the current list. To target a different chain, change the contract addresses and topic hashes to match the target chain. See the [Generator chain config](/run/run-a-generator/config-reference#chain) for details. ## Topology @@ -39,7 +39,7 @@ nginx terminates incoming HTTPS traffic and proxies GraphQL and metrics requests ## Config file -This config is drawn from `host-prod-setup.sh` in the `shinzo-host-client` repo. It sets the DefraDB URL, keyring secret, P2P bootstrap peers, and the ShinzoHub endpoint. The event filter is disabled, accepting all documents: +This config is drawn from `host-prod-setup.sh` in the `shinzo-host-client` repo. It sets the DefraDB URL, keyring secret, P2P bootstrap peers, and the ShinzoHub endpoint. The event filter is disabled and accepts all documents: ```yaml defradb: @@ -361,7 +361,7 @@ curl -s -X POST https://localhost/api/v0/graphql \ - The `DEFRA_KEYRING_SECRET` env var uses the `DEFRA_` prefix. The Generator client uses `DEFRADB_KEYRING_SECRET` with the `DEFRADB_` prefix. If you are running both clients on the same VM, do not confuse the two env var names. See [environment variables](/run/run-a-host/config-reference#environment-variables). - The shipped `config.yaml` includes several `shinzo.*` keys that are not in the `config.go` struct and are silently ignored: `wait_for_gaps`, `max_gap_size`, `batch_processing_enabled`, `batch_max_views_per_job`, `batch_query_cache_size`. They have been omitted from the config above. See [no-op keys](/run/run-a-host/config-reference#no-op-keys). - The `logger.level` field in the shipped config is not in the `LoggerConfig` struct and has no effect. It has been omitted. See [logger](/run/run-a-host/config-reference#logger). -- The bootstrap peer IDs in this config are the three indexer peers from `host-prod-setup.sh` (identical to the shipped `config.yaml`) and may be stale. Check the [Shinzo Validators list](https://registration.shinzo.network/validators) for current peers. +- The bootstrap peer IDs in this config are the three trustless indexer peers from `host-prod-setup.sh` (identical to the shipped `config.yaml`) and may be stale. Check the [Shinzo Validators list](https://registration.shinzo.network/validators) for current peers. - The nginx CORS config restricts origins to `https://explorer.shinzo.network`. If you need to allow other origins, add them to the `map` block in the nginx config. - The Host image tag in this compose file is `ghcr.io/shinzonetwork/shinzo-host-client:v0.6.5-ethereum-mainnet`, matching `host-prod-setup.sh`. The [GCP local SSD scenario](../gcp-local-ssd-raid0/) pins `:v0.5.1`, and the [Watchtower setup](../watchtower-auto-deploy/) uses `:latest`. Pick one tag and be consistent across your deployment. diff --git a/content/run/run-a-host/deployment-examples/snapshot-bootstrap/index.md b/content/run/run-a-host/deployment-examples/snapshot-bootstrap/index.md index 49ccbe1..5579446 100644 --- a/content/run/run-a-host/deployment-examples/snapshot-bootstrap/index.md +++ b/content/run/run-a-host/deployment-examples/snapshot-bootstrap/index.md @@ -1,14 +1,14 @@ +++ -title = "Snapshot bootstrap from an indexer" -description = "Bootstrap a Host with historical data on first startup by downloading signed snapshot files from an indexer over HTTPS, then receive live blocks over P2P." +title = "Snapshot bootstrap from a trustless indexer" +description = "Bootstrap a Host with historical data on first startup by downloading signed snapshot files from a trustless indexer over HTTPS, then receive live blocks over P2P." aliases = ["/hosts/deployment-examples/snapshot-bootstrap"] [extra] mermaid = true +++ -When to use this: you want your Host to sync historical data quickly on first startup by downloading signed snapshot files from an indexer, instead of waiting for P2P replication to catch up from the chain tip. +When to use this: you want your Host to sync historical data quickly on first startup by downloading signed snapshot files from a trustless indexer, instead of waiting for P2P replication to catch up from the chain tip. -These scenarios use data from a supported EVM chain. Shinzo supports multiple EVM chains — see [shinzo.network/chains](https://shinzo.network/chains) for the current list. To target a different chain, change the contract addresses and topic hashes to match the target chain. See the [Generator chain config](/run/run-a-generator/config-reference#chain) for details. +These scenarios use data from a supported blockchain. Shinzo supports multiple blockchains. See [shinzo.network/chains](https://shinzo.network/chains) for the current list. To target a different chain, change the contract addresses and topic hashes to match the target chain. See the [Generator chain config](/run/run-a-generator/config-reference#chain) for details. ## Topology @@ -30,17 +30,17 @@ flowchart LR Gens -- "P2P (libp2p)" --> Host {% end %} -On first startup, the Host downloads signed snapshot files from an indexer over HTTPS and imports them into DefraDB. After the snapshot import completes, the Host connects to Generators over P2P and starts receiving live blocks from the chain tip. This is faster than waiting for P2P replication to fill in historical data block by block. +On first startup, the Host downloads signed snapshot files from a trustless indexer over HTTPS and imports them into DefraDB. After the snapshot import completes, the Host connects to Generators over P2P and starts receiving live blocks from the chain tip. This is faster than waiting for P2P replication to fill in historical data block by block. ## Prerequisites - Docker installed on the VM. -- An indexer serving snapshots over HTTP. The indexer must have `SNAPSHOT_ENABLED=true` and be reachable over HTTPS or HTTP. See the [nginx with TLS scenario](/run/run-a-generator/deployment-examples/nginx-tls-snapshots/) for how to set up an indexer that serves snapshots. -- The block range you want to bootstrap. The indexer must have snapshot files covering that range. +- A trustless indexer serving snapshots over HTTP. The trustless indexer must have `SNAPSHOT_ENABLED=true` and be reachable over HTTPS or HTTP. See the [nginx with TLS scenario](/run/run-a-generator/deployment-examples/nginx-tls-snapshots/) for how to set up a trustless indexer that serves snapshots. +- The block range you want to bootstrap. The trustless indexer must have snapshot files covering that range. ## Config file -This config enables snapshot bootstrap from the indexer at `http://35.206.105.60:8080` for blocks 24528700 through 24528999. It is drawn from the `host.snapshot` section of the shipped `host-client/config/config.yaml`: +This config enables snapshot bootstrap from the trustless indexer at `http://35.206.105.60:8080` for blocks 24528700 through 24528999. It is drawn from the `host.snapshot` section of the shipped `host-client/config/config.yaml`: ```yaml defradb: @@ -101,7 +101,7 @@ host: ### What each snapshot value does - `host.snapshot.enabled: true`: Run snapshot bootstrap on startup. The shipped `config.yaml` sets this to false. See [host snapshot](/run/run-a-host/config-reference#host-snapshot). -- `host.snapshot.indexer_url`: HTTP base URL of the indexer serving snapshots. Replace `http://35.206.105.60:8080` with your indexer URL. See [host snapshot](/run/run-a-host/config-reference#host-snapshot). +- `host.snapshot.indexer_url`: HTTP base URL of the trustless indexer serving snapshots. Replace `http://35.206.105.60:8080` with your trustless indexer URL. See [host snapshot](/run/run-a-host/config-reference#host-snapshot). - `host.snapshot.historical_ranges`: Block ranges to download during bootstrap. Each range is inclusive. See [host snapshot](/run/run-a-host/config-reference#host-snapshot). ## How bootstrap works @@ -138,8 +138,8 @@ You should see log lines indicating snapshot downloads and imports, followed by ## Gotchas - Snapshot bootstrap only runs on first startup when DefraDB has no existing data. If the Host already has data for the requested block range, bootstrap is skipped. -- The `indexer_url` in the shipped `config.yaml` is `http://35.206.105.60:8080`. This is a development indexer. Replace it with your own indexer URL or a production indexer that has snapshots enabled. -- The indexer must have `SNAPSHOT_ENABLED=true` on the Generator side. If the indexer is not producing snapshot files, the `/snapshots` endpoint will return nothing and bootstrap will fail. See the [nginx with TLS scenario](/run/run-a-generator/deployment-examples/nginx-tls-snapshots/) for setting up an indexer that serves snapshots. +- The `indexer_url` in the shipped `config.yaml` is `http://35.206.105.60:8080`. This is a development trustless indexer. Replace it with your own trustless indexer URL or a production trustless indexer that has snapshots enabled. +- The trustless indexer must have `SNAPSHOT_ENABLED=true` on the Generator side. If the trustless indexer is not producing snapshot files, the `/snapshots` endpoint will return nothing and bootstrap will fail. See the [nginx with TLS scenario](/run/run-a-generator/deployment-examples/nginx-tls-snapshots/) for setting up a trustless indexer that serves snapshots. - The `DEFRA_URL` env var overrides `defradb.url` at runtime and is read by the Host client (`config/config.go`). The `docker run` above does not set it, so the DefraDB URL comes from `defradb.url` in the YAML config. See [environment variables](/run/run-a-host/config-reference#environment-variables). - The `DEFRA_KEYRING_SECRET` env var uses the `DEFRA_` prefix. The Generator client uses `DEFRADB_KEYRING_SECRET` with the `DEFRADB_` prefix. The two clients use different env var names for the same concept. See [environment variables](/run/run-a-host/config-reference#environment-variables). - `LOG_LEVEL`, `LOG_SOURCE`, and `LOG_STACKTRACE` env vars appear in some deployment scripts but are not read by the Host client. They have been omitted from the `docker run` above. See [env vars that are not read](/run/run-a-host/config-reference#env-vars-that-are-not-read). diff --git a/content/run/run-a-host/deployment-examples/watchtower-auto-deploy/index.md b/content/run/run-a-host/deployment-examples/watchtower-auto-deploy/index.md index 4963902..f9a4a05 100644 --- a/content/run/run-a-host/deployment-examples/watchtower-auto-deploy/index.md +++ b/content/run/run-a-host/deployment-examples/watchtower-auto-deploy/index.md @@ -8,7 +8,7 @@ mermaid = true When to use this: you want pushes to `main` to automatically deploy to your Host VM. GitHub Actions builds and pushes the image to GHCR, then Watchtower on the VM detects the new image and restarts the container. -These scenarios use data from a supported EVM chain. Shinzo supports multiple EVM chains — see [shinzo.network/chains](https://shinzo.network/chains) for the current list. To target a different chain, change the contract addresses and topic hashes to match the target chain. See the [Generator chain config](/run/run-a-generator/config-reference#chain) for details. +These scenarios use data from a supported blockchain. Shinzo supports multiple blockchains. See [shinzo.network/chains](https://shinzo.network/chains) for the current list. To target a different chain, change the contract addresses and topic hashes to match the target chain. See the [Generator chain config](/run/run-a-generator/config-reference#chain) for details. ## Topology @@ -160,7 +160,7 @@ Replace `sha-abc1234` with the actual commit SHA tag you want to roll back to. ## Gotchas - The container must have the `com.centurylinklabs.watchtower.enable=true` label or Watchtower will ignore it. Verify with `docker inspect shinzo-host | grep watchtower`. -- `DEFRA_URL` in the `docker run` commands overrides `defradb.url` and is read by the Host client (`config/config.go`), binding the DefraDB API to `0.0.0.0:9181`. `LOG_LEVEL`, `LOG_SOURCE`, and `LOG_STACKTRACE` are also kept from the original DEPLOYMENT.md but are not read by the Host client and have no effect. See [env vars that are not read](/run/run-a-host/config-reference#env-vars-that-are-not-read). +- `DEFRA_URL` in the `docker run` commands overrides `defradb.url` and is read by the Host client (`config/config.go`) and binds the DefraDB API to `0.0.0.0:9181`. `LOG_LEVEL`, `LOG_SOURCE`, and `LOG_STACKTRACE` also come from the original DEPLOYMENT.md but are not read by the Host client and have no effect. See [env vars that are not read](/run/run-a-host/config-reference#env-vars-that-are-not-read). - The Host image tag `:latest` is used here. The [prod VM scenario](../prod-vm-nginx-tls/) uses `:v0.6.5-ethereum-mainnet`, and the [GCP local SSD scenario](../gcp-local-ssd-raid0/) pins `:v0.5.1`. If you use Watchtower with `:latest`, you get automatic updates. If you pin a specific tag, Watchtower will not detect new images. - Watchtower preserves the container's configuration (ports, volumes, env vars, labels) across restarts. It only replaces the image. If you need to change the container configuration, stop and remove it manually, then start a new container with the updated configuration. - The `DEFRA_KEYRING_SECRET` GitHub secret is used for tests in the CI pipeline, not for runtime. The runtime keyring secret comes from your `config.yaml` on the VM. See [defradb config](/run/run-a-host/config-reference#defradb). diff --git a/content/run/run-a-host/hardware-requirements/index.md b/content/run/run-a-host/hardware-requirements/index.md index e09288f..f3a8a4f 100644 --- a/content/run/run-a-host/hardware-requirements/index.md +++ b/content/run/run-a-host/hardware-requirements/index.md @@ -3,11 +3,11 @@ title = "Hardware requirements" aliases = ["/hosts/hardware-requirements"] +++ -These requirements are for the Host client. A Host does not run a blockchain node and has no archival mode, so it never needs the multi-terabyte storage that an execution client or archival Generator requires. It receives signed primitive data over P2P from Generator clients, applies Lens transforms, and serves Views to applications. +These requirements are for the Host client. A Host does not run a blockchain node and has no archival mode, so it never needs the multi-terabyte storage that a full node or archival Generator requires. It receives signed primitive data over P2P from Generator clients, applies Lens transforms, and serves Views to applications. -## Recommended hardware +## Recommended hardware -The Host client specific hardware requirements depending on which chain the Generator it's linked to is indexing. See [shinzo.network/chains](https://shinzo.network/chains) for currently networks. +The Host client has specific hardware requirements depending on the linked Generator's chain. See [shinzo.network/chains](https://shinzo.network/chains) for the current list of supported networks. ### Ethereum mainnet @@ -20,7 +20,7 @@ The Host client specific hardware requirements depending on which chain the Gene ## Storage -Host storage depends almost entirely on how many Views you serve and how aggressively you prune. A Host serving a few filtered Views with pruning enabled stays close to the minimum. A Host that accepts all primitive data and serves many materialized Views will trend toward the recommended figure and beyond. Pruning is enabled by default and retains roughly the last 2,000 blocks. Because the Host receives primitives from Generator clients, its storage growth tracks the throughput of the source chain — chains with higher transaction volume produce more documents per block. See [shinzo.network/chains](https://shinzo.network/chains) for the chains Shinzo supports. +Host storage depends almost entirely on how many Views you serve and how aggressively you prune. A Host serving a few filtered Views with pruning enabled stays close to the minimum. A Host that accepts all primitive data and serves many materialized Views will trend toward the recommended figure and beyond. Pruning is enabled by default and retains roughly the last 2,000 blocks. Because the Host receives primitives from Generator clients, its storage growth tracks the throughput of the source chain. Chains with higher transaction volume produce more documents per block. See [shinzo.network/chains](https://shinzo.network/chains) for the chains Shinzo supports. ## Network diff --git a/content/run/run-a-host/install/index.md b/content/run/run-a-host/install/index.md index e495d51..8cefa18 100644 --- a/content/run/run-a-host/install/index.md +++ b/content/run/run-a-host/install/index.md @@ -68,7 +68,7 @@ Pull the image and start it with a single `docker run`. You supply two values: a | --- | --- | --- | | `9181` | DefraDB GraphQL + REST API | | | `9182` | GraphQL Playground UI | | - | `9171` | libp2p P2P networking | Must be reachable from the internet — open/forward this port | + | `9171` | libp2p P2P networking | Must be reachable from the internet. Open or forward this port | | `8080` | Health + metrics | Served inside the container; publish with `-p 8080:8080` if you want to scrape it | 1. Confirm the container is up: @@ -89,7 +89,7 @@ Pull the image and start it with a single `docker run`. You supply two values: a A JSON response confirms the API is serving. On a freshly started Host client you'll see an error like `{"errors":[{"message":"key not found"}],"data":null}`, which is expected. The API is up; there's just no data yet. You can also open the Playground at `http://localhost:9182` to confirm the UI loads. -1. Returning real indexed data depends on the Host client reaching the Generator client you set in `BOOTSTRAP_PEERS` and syncing from it. Check the connection in the logs: +1. To return real data, the Host client must reach the Generator client you set in `BOOTSTRAP_PEERS` and sync from it. Check the connection in the logs: ```shell docker logs shinzo-host | grep -i peer @@ -153,7 +153,7 @@ make build-playground ### Configure and run -The only required setting is the keyring secret. +The only required setting is the keyring secret. 1. Export the secret key: @@ -189,6 +189,6 @@ The only required setting is the keyring secret. Ports, verification, and registration are the same as the Docker path above. -## Need Help +## Need help {{ need_help(client="Host", repo_name="shinzo-host-client", repo="https://github.com/shinzonetwork/shinzo-host-client/issues") }} diff --git a/content/run/run-a-host/private-hosts/index.md b/content/run/run-a-host/private-hosts/index.md index 38cf066..f19b903 100644 --- a/content/run/run-a-host/private-hosts/index.md +++ b/content/run/run-a-host/private-hosts/index.md @@ -27,7 +27,7 @@ A private Host client connection closes some or all of these. How many you close ## Two tiers -There are two tiers of privacy. Select that one that works for your use-case. +There are two tiers of privacy. Select the one that works for your use case. ### Standard private Host client @@ -35,7 +35,7 @@ Keep ShinzoHub connected so the Host client still fetches and runs the public Vi What changes from the defaults: -- `defradb.p2p.bootstrap_peers`: replace the public peers with your Generator client's multiaddr. Do no include any other multiaddrs. +- `defradb.p2p.bootstrap_peers`: replace the public peers with your Generator client's multiaddr. Do not include any other multiaddrs. - `shinzo.hub_base_url`: set it to `testnet.shinzo.network:26657` (the shipped value; the code default is empty) so the Host client keeps fetching public Views. - Skip [Register](/hosts/register/). An unregistered Host isn't discoverable and won't serve the network. @@ -141,6 +141,6 @@ Restart the same container, not a fresh one. `views.json` and the cached WASM le - Your Generator is your only data source. If it goes down or falls behind, the Host client has no public fallback in a fully air-gapped setup. - You manage View updates yourself. In a standard private host setup the hub still pushes new registrations. In a fully air-gapped setup, you re-run the ingest step to pick up new public Views. -## Need Help +## Need help {{ need_help(client="Host", repo_name="shinzo-host-client", repo="https://github.com/shinzonetwork/shinzo-host-client/issues") }} diff --git a/content/run/run-a-host/quickstart/index.md b/content/run/run-a-host/quickstart/index.md index 2fd91fa..31d8103 100644 --- a/content/run/run-a-host/quickstart/index.md +++ b/content/run/run-a-host/quickstart/index.md @@ -4,22 +4,22 @@ aliases = ["/hosts/quickstart"] description = "Run a Shinzo Host to transform blockchain data into verifiable Views" +++ -Hosts turn raw blockchain data into structured **Views** and produce **Attestation Records** that help secure the network. This guide covers installing, configuring, and running the Shinzo Host Client. +Hosts turn raw blockchain data into structured Views and produce Attestation Records that help secure the network. This guide covers installing, configuring, and running the Shinzo Host Client. ## Hardware recommendations -The Host client does not run a blockchain node and has no archival mode, so it never needs the multi-terabyte storage of an execution client or archival Generator. See the [hardware requirements page](../hardware-requirements/) for CPU, RAM, storage, and network sizing. +The Host client does not run a blockchain node and has no archival mode, so it never needs the multi-terabyte storage of a full node or archival Generator. See the [hardware requirements page](../hardware-requirements/) for CPU, RAM, storage, and network sizing. -## Local Deployment +## Local deployment Run the Shinzo Host Client directly on your local machine for development and testing. ### Prerequisites -- Go 1.25 -- Metamask with a wallet setup. This wallet does not need to hold any funds. +- Go 1.25. +- MetaMask with a wallet setup. This wallet does not need to hold any funds. -### Clone the Repository +### Clone the repository ```shell git clone https://github.com/shinzonetwork/shinzo-host-client.git @@ -28,17 +28,17 @@ cd shinzo-host-client ### Configuration -The Host Client reads from [config.yaml](https://github.com/shinzonetwork/shinzo-host-client/blob/main/config/config.yaml), which comes with working defaults. The only field you need to set is `defradb.keyring_secret`. Alternatively, you can also set the password as an environment variable to avoid storing it in plaintext: +The Host Client reads from [config.yaml](https://github.com/shinzonetwork/shinzo-host-client/blob/main/config/config.yaml), which comes with working defaults. The only field you need to set is `defradb.keyring_secret`. You can also set the password as an environment variable to avoid storing it in plaintext: ```shell export DEFRA_KEYRING_SECRET= ``` -#### Key Fields +#### Key fields - `defradb.url`: API endpoint of your local DefraDB node. Defaults work for most setups. - `defradb.keyring_secret`: Requires a secret to generate your private keys. -- `p2p.bootstrap_peers`: Generator client peers for receiving indexed data. Defaults include a reliable bootstrap peer. +- `p2p.bootstrap_peers`: Generator client peers for receiving verifiable data. Defaults include a reliable bootstrap peer. - `p2p.listen_addr`: Default is suitable for local runs. Override when containerizing. - `store.path`: Directory where local DefraDB data is stored. - `shinzo.web_socket_url`: Defaults to a hosted ShinzoHub node. Only change if connecting to a different node. @@ -63,20 +63,20 @@ If you are running a Generator client and a Host client on the same machine, app The Generator client is likely already using port `9181`, so update the `defradb.url` field: -```shell +```yaml url: "localhost:9182" ``` Also update the [P2P settings](https://github.com/shinzonetwork/shinzo-host-client/blob/main/config/config.yaml#L4) to use localhost and a different port so the Host doesn't clash with the Generator client: -```shell +```yaml bootstrap_peers: - '/ip4/127.0.0.1/tcp/9171/p2p/' listen_addr: "/ip4/0.0.0.0/tcp/9172" ``` -### Build and Run +### Build and run {{ tab(label="Run directly") }} ```shell @@ -129,7 +129,7 @@ query GetLatestLogs { More query examples are available [here](/build/query-data/). -## VM Deployment +## VM deployment This is the recommended approach for production and testnet participation. It uses Docker, docker-compose, and Nginx on a virtual machine. @@ -137,21 +137,21 @@ This is the recommended approach for production and testnet participation. It us - Port `444` open in your firewall/security group. -### Install System Dependencies +### Install system dependencies ```shell sudo apt-get update sudo apt-get install -y docker.io docker-compose nginx ``` -### Create the Data Directory +### Create the data directory ```shell sudo mkdir -p ~/data/defradb ~/data/lens sudo chown -R 1001:1001 ~/data/defradb ~/data/lens ``` -### Generate SSL Certificates +### Generate SSL certificates ```shell # Generate private key, certificate signing request, and self-signed certificate @@ -163,7 +163,7 @@ sudo openssl x509 -req -days 365 -in /tmp/nginx.csr -signkey ~/ssl/nginx.key -ou sudo rm /tmp/nginx.csr ``` -### Write the Configuration File +### Write the configuration file Create `~/config.yaml`. The production config enables performance tuning, peer reconnection, pruning, and optional event filtering. Key values to set: @@ -194,11 +194,11 @@ host: The full production config is generated automatically by `host-prod-setup.sh`. See below. {% end %} -### Write the Nginx Config +### Write the Nginx config Create `~/nginx.conf`: -```shell +```nginx events { worker_connections 1024; } http { @@ -237,8 +237,8 @@ http { } } ``` - -### Write the docker-compose File + +### Write the docker-compose file Create `~/docker-compose.yml`: @@ -301,7 +301,7 @@ docker-compose up -d The health check endpoint is available at: -```shell +```plaintext http://:8080/metrics ``` @@ -312,35 +312,35 @@ docker ps docker logs shinzo-host ``` -### Docker Image +### Docker image The multi-stage Dockerfile builds the host binary (Go 1.25) along with the Wasmtime and Wasmer WASM runtimes. The production image is based on Ubuntu 24.04 and runs as a non-root `shinzo` user. Pre-built images are published to: -```shell +```plaintext ghcr.io/shinzonetwork/shinzo-host-client:ethereum-mainnet-latest ``` -## ShinzoHub Registration +## ShinzoHub registration To participate in the Shinzo Network, you must register your Host. Registration identifies your node so it can replicate data and earn rewards. An unregistered Host will not be recognized by the network. There are two ways to register: -### Option A: Register with the GUI +### Register with the GUI (Option A) 1. Start your Host client. 1. Add Shinzo testnet to Metamask with the following values: - Network name: Shinzo - - Default RPC URL: http://testnet.shinzo.network:8545 - - Chain ID: 91273001 - - Currency symbol: SHNZ + - Default RPC URL: `http://testnet.shinzo.network:8545` + - Chain ID: `91273001` + - Currency symbol: `SHNZ` You need a small amount of SHNZ for the registration transaction fee. Get testnet SHNZ from the [faucet](https://faucet.shinzo.network/). 1. Open the [Registration app](https://registration.shinzo.network/) and connect your wallet. If your Host is on a private network, the Host client also serves a local registration app at `http://localhost:8080/registration-app`; use SSH port forwarding to reach a remote node. 1. On the registration page, click **Register** and select **Host** as your role to complete the process. 1. Submit your registration, then confirm the transaction in MetaMask. You should see a successful registration notification. -### Option B: Register with the CLI +### Register with the CLI (Option B) -You can also register your Host by submitting the registration transaction directly with Foundry’s `cast` CLI. +You can also register your Host by submitting the registration transaction directly with Foundry's `cast` CLI. ```shell cast send "0x0000000000000000000000000000000000000211" \ @@ -365,10 +365,10 @@ Be careful with your private key. Do not commit it to source control, paste it i Your Host is now registered and authorized to participate in the Shinzo Network. -## Need Help +## Need help {{ need_help(client="Host", repo_name="shinzo-host-client", repo="https://github.com/shinzonetwork/shinzo-host-client/issues") }} -## Next Steps +## Next steps Your Host can now receive and serve Views. A registered Host starts serving a specific View by joining its [pool](/understand/core-concepts/pools/). That's what commits a Host to running a particular View. Try running queries against it through the playground GUI. diff --git a/content/run/run-a-host/register/index.md b/content/run/run-a-host/register/index.md index 676df78..9c68b90 100644 --- a/content/run/run-a-host/register/index.md +++ b/content/run/run-a-host/register/index.md @@ -7,10 +7,12 @@ To participate in the Shinzo Network and make your View publicly available, you ## Add the Shinzo Testnet to your wallet -- Network name: `Shinzo` -- Default RPC URL: `http://testnet.shinzo.network:8545` -- Chain ID: `91273001` -- Currency symbol: `SHNZ` +| Field | Value | +| --- | --- | +| Network name | `Shinzo` | +| Default RPC URL | `http://testnet.shinzo.network:8545` | +| Chain ID | `91273001` | +| Currency symbol | `SHNZ` | ## Register with the hosted app @@ -34,7 +36,7 @@ If your Host is on a private network or you'd rather not route registration thro 1. Open `http://localhost:8080/registration-app` in your browser. {% end %} -1. Click **Register as Host** and fill out all the details +1. Click **Register as Host** and fill out all the details. 1. Submit your registration and confirm the transaction in your browser wallet. You should see a successful registration notification. ## Register with the CLI @@ -53,6 +55,6 @@ This key is your Host's identity on the network. If you lose it without a backup If this key is lost with no backup available, you will need to spin up a new Host and re-register with a new identity. {% end %} -## Need Help +## Need help {{ need_help(client="Host", repo_name="shinzo-host-client", repo="https://github.com/shinzonetwork/shinzo-host-client/issues") }} From ad0937daeeee47068a30a030143d6a7f374a5e93 Mon Sep 17 00:00:00 2001 From: johnnymatthews <9611008+johnnymatthews@users.noreply.github.com> Date: Thu, 20 Aug 2026 12:06:10 +0100 Subject: [PATCH 4/5] Tidies up understand section. --- content/understand/core-concepts/_index.md | 8 ++-- .../core-concepts/attestation/index.md | 2 +- .../understand/core-concepts/privacy/index.md | 6 +-- content/understand/how-it-works/index.md | 10 ++--- content/understand/what-is-shinzo/index.md | 38 +++++++++++-------- 5 files changed, 35 insertions(+), 29 deletions(-) diff --git a/content/understand/core-concepts/_index.md b/content/understand/core-concepts/_index.md index 60d33f3..879cebe 100644 --- a/content/understand/core-concepts/_index.md +++ b/content/understand/core-concepts/_index.md @@ -6,7 +6,7 @@ Shinzo runs on four ideas that show up everywhere in the stack: Views, Attestati Each concept has its own page below: -- [Views](views/) — what a View is, what it contains, and why it's designed the way it is. -- [Attestations](attestation/) — how Shinzo tracks how much of the network has independently agreed on a piece of data. -- [DefraDB](defradb/) — the peer-to-peer document database embedded in every component. -- [Tokenomics](shnz-token/) — the economic unit flowing between developers, Hosts, and consumers. +- [Views](views/): what a View is, what it contains, and why it's designed the way it is. +- [Attestations](attestation/): how Shinzo tracks how much of the network has independently agreed on a piece of data. +- [DefraDB](defradb/): the peer-to-peer document database embedded in every component. +- [Tokenomics](shnz-token/): the economic unit flowing between developers, Hosts, and consumers. diff --git a/content/understand/core-concepts/attestation/index.md b/content/understand/core-concepts/attestation/index.md index b08512e..32bf059 100644 --- a/content/understand/core-concepts/attestation/index.md +++ b/content/understand/core-concepts/attestation/index.md @@ -5,7 +5,7 @@ Attestation is how Shinzo tracks how much of the network has independently agree ## The problem it solves -With a centralized indexing service, you trust the provider because you have no other option. You can't verify[^1] the data you got matches what's actually on chain. In Shinzo, Generators cryptographically sign every document they produce, so there's a verifiable record of who said what. But a single signature only goes so far. You still need to know whether multiple independent Generators saw the same thing. +With a centralized data service, you trust the provider because you have no other option. You can't verify[^1] the data you got matches what's actually on chain. In Shinzo, Generators cryptographically sign every document they produce, so there's a verifiable record of who said what. But a single signature only goes so far. You still need to know whether multiple independent Generators saw the same thing. Attestation answers that question. diff --git a/content/understand/core-concepts/privacy/index.md b/content/understand/core-concepts/privacy/index.md index d81d9d9..df1fe97 100644 --- a/content/understand/core-concepts/privacy/index.md +++ b/content/understand/core-concepts/privacy/index.md @@ -1,6 +1,6 @@ +++ title = "Privacy" -description = "If your concerned about keeping all your Views and queries private, you may want to consider spinning up a private Host client, sometimes called a Direct Client." +description = "If you're concerned about keeping all your Views and queries private, you may want to consider spinning up a private Host client, sometimes called a Direct Client." mermaid = true +++ @@ -25,7 +25,7 @@ A private Host client connection closes some or all of these. How many you close ## Two tiers -There are two tiers of privacy. Select that one that works for your use-case. +There are two tiers of privacy. Select the one that works for your use-case. ### Standard private Host client @@ -33,7 +33,7 @@ Keep ShinzoHub connected so the Host client still fetches and runs the public Vi What changes from the defaults: -- `defradb.p2p.bootstrap_peers`: replace the public peers with your Generator client's multiaddr. Do no include any other multiaddrs. +- `defradb.p2p.bootstrap_peers`: replace the public peers with your Generator client's multiaddr. Do not include any other multiaddrs. - `shinzo.hub_base_url`: set it to `testnet.shinzo.network:26657` (the shipped value; the code default is empty) so the Host client keeps fetching public Views. - Skip [Register](/hosts/register/). An unregistered Host isn't discoverable and won't serve the network. diff --git a/content/understand/how-it-works/index.md b/content/understand/how-it-works/index.md index a3e786e..06270df 100644 --- a/content/understand/how-it-works/index.md +++ b/content/understand/how-it-works/index.md @@ -5,12 +5,12 @@ aliases = ["/introduction/how-it-works"] mermaid = true +++ -Shinzo has four kinds of moving parts: +Shinzo has four kinds of moving parts: 1. **Generators** that read the chain. -1. **Hosts** that transform and serve the data. +1. **Hosts** that transform and serve the data. 1. **Applications** that consume that data. -1. **ShinzoHub**, a coordination layer that tells everyone what's going on. +1. **ShinzoHub**, a coordination layer that tells everyone what's going on. Data flows from left to right. Coordination happens on the side. @@ -183,7 +183,7 @@ Applications embed DefraDB locally, subscribe to the Views they need, and query ShinzoHub is a Cosmos SDK chain that sits to the side of the data path. It doesn't carry bulk data itself (that all flows over the P2P network between DefraDB instances). What ShinzoHub does is keep the registry of who's on the network and what they can do. -That means three things in practice: +That means three things in practice: 1. **View registration**: When a developer deploys a View, ShinzoHub validates and registers it, then emits an event that Hosts listen for. 1. **Participant tracking**: Generators and Hosts register themselves on-chain so the rest of the network can discover them. @@ -201,6 +201,6 @@ Validators already run full nodes, already have the block data the moment it's p Generator clients ingest, Host clients transform and serve. That split means Generator clients can stay small and cheap (so validators will actually run them), while Host clients can specialize. One Host client might process every DeFi View on the network, another might focus on NFTs. It also means scaling consumer demand is a matter of adding more Host clients, not Generator clients. -### Apps query local data, not remote APIs +### Apps query local data not remote APIs Because application clients embed DefraDB and receive pre-processed view data over P2P, a query is a local database lookup. You don't pay per read, you don't hit rate limits, and you can verify what you got against the attestation record before you trust it. The trade-off is that your app is _pushed_ data for the Views it subscribes to rather than pulling arbitrary slices. diff --git a/content/understand/what-is-shinzo/index.md b/content/understand/what-is-shinzo/index.md index 0e96e73..c4c6cd1 100644 --- a/content/understand/what-is-shinzo/index.md +++ b/content/understand/what-is-shinzo/index.md @@ -6,27 +6,33 @@ page_template = "page.html" mermaid = true +++ -Shinzo is a decentralized indexing network for blockchains. It takes raw on-chain data and turns it into structured datasets that any application can query, without having to go through a centralized indexing service to get them. +Shinzo is a trustless network for reading blockchain data. It takes raw on-chain data and turns it into structured datasets that any application can query, without having to go through a centralized data service to get them. -If you've built any kind web3 app before, you know the usual pattern: you pick a centralized service provider (alchemy, infura), pay per API call, cache the results locally, and hope the provider doesn't go down or quietly change what's available. Shinzo replaces that setup with a network of independent operators that index the chain at the source and share the results peer to peer. +If you've built any kind web3 app before, you know the usual pattern: you pick a centralized service provider (alchemy, infura), pay per API call, cache the results locally, and hope the provider doesn't go down or quietly change what's available. Shinzo replaces that setup with a network of independent operators that read the chain at the source and share the results peer to peer. ## The problem Shinzo solves -Blockchains are good at writing data and bad at reading it. If you want to show a user all their previous transactions, or count token transfers for a given contract, you can't just ask the chain. The raw data isn't organized for questions like that. So the industry bolted centralized indexing services onto the side of every chain, and those services now sit in the trust path between your app and the data. +Blockchains are good at writing data and bad at reading it. If you want to show a user all their previous transactions, or count token transfers for a given contract, you can't just ask the chain. The raw data isn't organized for questions like that. So the industry bolted centralized data services onto the side of every chain, and those services now sit in the trust path between your app and the data. -This setup is expensive and fragile. The indexer's DNS might fail, or the cloud service hosting it might go down. But the biggest failure of this system is that you there's no way to verify that the data you're receiving is accurate until _after_ you've received (and paid) for it. +This setup is expensive and fragile. The provider's DNS might fail, or the cloud service hosting it might go down. But the biggest failure of this system is that there's no way to verify that the data you're receiving is accurate until _after_ you've received (and paid) for it. -The goal of Shinzo is to make reading blockchain data as decentralized and verifiable as writing to it. +The goal of Shinzo is to make reading blockchain data as trustless and verifiable as writing to it. ## How it works Three kinds of participants run the network. -**Generator clients** sit next to blockchain nodes and turn new blocks into structured documents as they arrive. They cryptographically sign everything they produce, so anyone downstream can check that the data is valid. +### Generator clients -**Host clients** receive primitive data from Generator clients over a peer-to-peer network and apply user-defined transforms called _Views_. They keep attestation records, which count how many different Generator clients signed off on each piece of data, so developers can use those counts to set their own trust thresholds. +Generator clients sit next to blockchain nodes and turn new blocks into structured documents as they arrive. They cryptographically sign everything they produce, so anyone downstream can check that the data is valid. -**Developers** define the Views. A View is basically a way to say _"here's the raw data I care about, here's how I want it filtered and decoded, and here's how I want it structured."_ Once a View is deployed, Hosts pick it up, run it, and push the results to whoever subscribes to that particular View. +### Host clients + +Host clients receive primitive data from Generator clients over a peer-to-peer network and apply user-defined transforms called _Views_. They keep attestation records, which count how many different Generator clients signed off on each piece of data, so developers can use those counts to set their own trust thresholds. + +### Developers + +Developers define the Views. A View is basically a way to say _"here's the raw data I care about, here's how I want it filtered and decoded, and here's how I want it structured."_ Once a View is deployed, Hosts pick it up, run it, and push the results to whoever subscribes to that particular View. {% mermaid() %} flowchart LR @@ -82,7 +88,7 @@ If you're a developer working on an app, wallet, or any kind of web3 service, Vi - A schema (GraphQL SDL) describing how you want that data organized. - Lens transforms to filter, decode, and shape that data into what you _actually_ need. -You build Views with `viewkit`, Shinzo's CLI, and deploy them to Shinzohub so that anyone else use them. If you'd rather skip the CLI, [Shinzo Studio](https://studio.shinzo.network/) provides the same build-and-deploy workflow in the browser. Any Host client can then pick up the View, run it, and serve the results. Your application subscribes through the [app-sdk](https://github.com/shinzonetwork/app-sdk) and queries the resulting data locally. +You build Views with `viewkit`, Shinzo's CLI, and deploy them to ShinzoHub so that anyone else can use them. If you'd rather skip the CLI, [Shinzo Studio](https://studio.shinzo.network/) provides the same build-and-deploy workflow in the browser. Any Host client can then pick up the View, run it, and serve the results. Your application subscribes through the [app-sdk](https://github.com/shinzonetwork/app-sdk) and queries the resulting data locally. ## Where the project is today @@ -90,12 +96,12 @@ Shinzo's public testnet is now live. Anyone can join the network by running the The current testnet includes: -- The **Generator** client reads a supported chain from an execution node, signs the data, and replicates it across the network using DefraDB's libp2p-based replication layer. See [shinzo.network/chains](https://shinzo.network/chains) for the current list of supported chains. -- The **Host** client receives replicated data from Generators, materializes registered Views using Lens transforms, and serves GraphQL queries to applications. -- **Viewkit** allows developers to define, package and deploy custom Views to the network, making indexed datasets immediately available to participating Hosts. -- **ShinzoHub** coordinates network participation, View registration, entity registration, and access control for the testnet. -- The **Explorer** lets anyone browse transactions and other on-chain activity across the testnet without running a node. -- The **Gateway** provides a unified GraphQL endpoint by routing requests across Hosts and validating responses through network consensus. -- The **App SDK** enables Go applications to embed Shinzo components and query attestation-filtered data directly from the network. +- The Generator client reads a supported chain from an execution node, signs the data, and replicates it across the network using DefraDB's libp2p-based replication layer. See [shinzo.network/chains](https://shinzo.network/chains) for the current list of supported chains. +- The Host client receives replicated data from Generators, materializes registered Views using Lens transforms, and serves GraphQL queries to applications. +- Viewkit allows developers to define, package and deploy custom Views to the network, making the datasets immediately available to participating Hosts. +- ShinzoHub coordinates network participation, View registration, entity registration, and access control for the testnet. +- The Explorer lets anyone browse transactions and other on-chain activity across the testnet without running a node. +- The Gateway provides a unified GraphQL endpoint by routing requests across Hosts and validating responses through network consensus. +- The App SDK enables Go applications to embed Shinzo components and query attestation-filtered data directly from the network. This testnet is intended to validate Shinzo's trustless indexing architecture under real-world conditions, gather feedback from operators and developers and harden the protocol ahead of mainnet. From 8f768383bfb58a6d84bab30de1527b26c5e8c1dd Mon Sep 17 00:00:00 2001 From: Johnny <9611008+johnnymatthews@users.noreply.github.com> Date: Thu, 20 Aug 2026 12:09:46 +0100 Subject: [PATCH 5/5] Tiny markdown edits I missed first time. Co-authored-by: Johnny <9611008+johnnymatthews@users.noreply.github.com> --- content/build/build-an-app/index.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/content/build/build-an-app/index.md b/content/build/build-an-app/index.md index 7b81ffc..8f0f48a 100644 --- a/content/build/build-an-app/index.md +++ b/content/build/build-an-app/index.md @@ -21,7 +21,7 @@ Shinzo uses [DefraDB](https://github.com/sourcenetwork/defradb) for several purp Consider a simple app to illustrate how Shinzo works. The app displays a counter for the current number of instances of a specified ERC-20 token, such as USDC. For argument's sake, assume the contract has no method to query the current supply of USDC. The only way to determine the supply is to parse the mint and burn events emitted by the contract. -To do this, you first create a View that describes how to transform primitive data (blocks, logs, transactions, etc.) into a format you can use. In this case, you filter logs involving the USDC contract address, decode the logs into events using the contract's ABI, and finally filter for only mint and burn events. The Shinzo Hosts and Generator clients work together to deliver the data you need. Your application client(s) receive all the mint and burn events on that USDC contract. From here, you can make as many GraphQL queries against the events you've received to build your application. Your app client(s) won't receive the underlying primitives (blocks, transactions, logs, etc.), only the filtered and decoded events as described in your View. +To do this, you first create a View that describes how to transform primitive data (blocks, logs, transactions, etc.) into a format you can use. In this case, you filter logs involving the USDC contract address, decode the logs into events using the contract's ABI, and finally filter for only mint and burn events. The Shinzo Host and Generator clients work together to deliver the data you need. Your application client(s) receive all the mint and burn events on that USDC contract. From here, you can make as many GraphQL queries against the events you've received to build your application. Your app client(s) won't receive the underlying primitives (blocks, transactions, logs, etc.), only the filtered and decoded events as described in your View. ## Usage @@ -45,7 +45,6 @@ This sets the default minimum attestations required when querying your Views. Se ```yaml logger: development: true -``` This enables all logs. If excluded, it defaults to false and silences most of the Defra logs. Setting development to false (or omitting it) is recommended for production, since Defra produces a lot of logs otherwise. Config can be handled in two ways. You can create the config options by hand. The [app-sdk creates a default config](https://github.com/shinzonetwork/app-sdk/blob/main/pkg/defra/defra.go#L23) this way, which is used in place of a nil config. You can also create a config.yaml file ([example](https://github.com/shinzonetwork/app-sdk/blob/main/config.yaml)) and load it with `config.LoadConfig`. To locate your config.yaml file, the `file.FindFile` helper is useful, especially in a test context. For example: