New features, performance improvements, and fixes across every HyperIndex release.
pnpm install envio@latestgetWhereawait context.Account.getWhere({
id: {
_eq: "0x123...",
},
balance: {
_gte: 1_000_000n,
_lte: 10_000_000n,
},
});
Additionally, we noticeably improved the performance of getWhere with a single _eq or _in filter by optimizing the number of round trips to the database.
Now it's possible to mark storages as default to automatically assign an entity to the storage. It allows you not to set @storage attribute for every entity in schema.graphql and only use it as a default overwrite.
storage:
postgres:
default: true
clickhouse:
default: true
Configure automatic conversion for column names in your database to snake_case. It only affects the underlying database, so GraphQL and handler types will preserve the original names derived from schema.graphql.
storage:
postgres:
column_name_format: snake_case
We added support for HyperSync-powered instruction handlers. Reach out to us if you're interested in becoming an early tester.
We are yet to reach our full potential. And the new HyperIndex release comes with a nice improvement: it requires up to 2x fewer HyperSync queries for backfill and is 2.5x faster in many indexing cases.
Describe your entities, fields, and relationships in the schema.graphql file, so that they are exposed in the GraphQL API.
# Hash comments are ignored by the GraphQL parser and should NOT appear
# in introspection. Only string descriptions ("""...""" or "...") are exposed.
"An on-chain account participating in transfers"
type Account {
"The account's Ethereum address"
id: ID!
"All transfers sent from this account"
outgoing: [Transfer!]! @derivedFrom(field: "from")
}
envio tools search-docs <query> subcommand #1237envio tools fetch-docs <url> subcommand #1237envio metrics runtime subcommand #1281envio init and envio skills update #1234Improve HyperSync rate-limit handling by showing rate-limit information in TUI and logs.
Excludes the chain from indexing and migrations by adding the skip field to the chain in the config.yaml file.
This is an advanced feature. For testing, prefer a test framework instead.
chains:
- id: 1
skip: true
It's production-ready, fast, and reliable!
We had an exciting journey with new features and improvements, and now that the five months of alpha releases are over, we are back to following SemVer. There are a number of breaking changes in the V3 release, but they will allow us to accelerate HyperIndex development and bring you even more features, performance, and reliability improvements.
We have big plans for HyperIndex to make it the ultimate tool for interacting with blockchain data, and you can help us build it: create GitHub Issues, reach out to us with feedback, and support us with a GitHub Star and a kind word to a friend.
As for the most recent roadmap we have:
Effect API, released on May 8, served us well, and we officially removed the experimental_ prefix from createEffect.
This change comes with two new features:
rateLimit option (required) which allows you to set explicitely how freaquently the effect might be called. Use either false to disable, or provide an object with custom calls per duration. Duration might be second, minute or a number in milliseconds.cache option for specific call. Set context.cache = false to prevent caching the result which failed.export const getMetadata = createEffect(
{
name: "getMetadata",
input: S.string,
output: S.optional(S.schema({
description: S.string,
value: S.bigint,
})),
// Protect your API from burst Effect calls
rateLimit: {
calls: 5,
per: "second"
},
cache: true,
},
async ({ input, context }) => {
try {
const response = await fetch(`https://api.example.com/metadata/${input}`);
const data = await response.json();
return {
description: data.description,
value: data.value,
};
} catch(_) {
// Don't cache failed response
context.cache = false
return undefined;
}
}
);
With this version, the Development Console now provides more performance metrics for Effect API execution, as well as loading entities from your handlers.
See a detailed time overview to identify what's slowing down your processing time. Use Batched metric to find what can be better optimized with Preload Optimization. And get useful insigts about your indexer execution in general. As a nice bonus, the Development Console now works even with TUI disabled.
Tip: Hover over question marks to get better understanding of the data.
<img width="620" height="753" alt="image" src="https://github.com/user-attachments/assets/631fdf7b-8367-45a6-aedc-a2148d3fcc0c" />Access Development Console at envio.dev/console
getWhere.lt QueryUse context.<Entity>.getWhere.<FieldName>.lt to get all entities where the field value is lower than the given value.
We completely rewrote our rollback on reorg logic to make it more robust, predictable, and performant. The update introduces numerous new tests and fixes for all previously identified indexing issues:
context.chains #775envio_progress_batches_countcontext access after the handler finishes execution #791_history with prefix envio_history_end_of_block_range_scanned_data table with envio_checkpoints
Choose between checksum (default) and lowercase addresses.
The newly added lowercase format can facilitate migration from SubGraphs and improve performance in some instances.
RPC source now uses the HyperSync event decoder, which is multiple times faster than the Viem one.
These changes are preparations for a bigger rollback on reorg refactoring.
Run logic on every block or at an interval!
This is useful for aggregations, time-series logic, and bulk updates using raw SQL. To get started, import the onBlock function from the generated module and call it in one of your handler files.
import { onBlock } from "generated";
onBlock(
{
name: "MyBlockHandler",
chain: 1,
interval: 10,
startBlock: 10_000_000,
},
async ({ block, context }) => {
context.log.info(`Processing block ${block.number}`);
}
);
Read a dedicated documentation page to learn more about the feature and powerful use cases it enables:
For inspiration, check out the Example Indexer, which uses all the latest Envio features - Block handlers with different intervals, Effect API, and Traces indexing to get all contracts deployed on Mainnet.
db_write_timestamp field, which was automatically set for every entity._meta queryReturns indexing metadata per chain, which helps track the indexer's progress.
<img width="991" height="528" alt="image" src="https://github.com/user-attachments/assets/60f82026-e659-4c16-9d5b-1d0acca5bf6a" />Improved block range selection for logs query, halving the number of needed requests for some RPC providers.
Optimized response from HyperSync to make queries simpler, faster, and have smaller ingress.
Added a performance optimization for indexers with a large number of addresses (>500k). It reduced the indexing time of an indexer with 2 million addresses from 4 days to 2.
Find a Cursor rule example in our repo to help you effortlessly migrate your existing SubGraph to HyperIndex!
cache from .gitignore for initial templatesThe release initiates a significant refactoring of internal tables, focusing on a single public entry point - _meta.
To achieve this, we hide all internal tables from Hasura, as well as change the internal representation of some of them. These are: chain_metadata, event_sync_state, persisted_state, end_of_block_range_scanned_data, and dynamic_contract_registry. Please let us know if this change affected you and we'll figure out a solution
Important! Preload optimization makes your handlers run twice. Starting from
envio@2.27all new indexers are created with preload optimization pre-configured by default.
This optimization enables HyperIndex to efficiently preload entities used by handlers through batched database queries, while ensuring events are processed synchronously in their original order. When combined with the Effect API for external calls, this feature delivers performance improvements of multiple orders of magnitude compared to other indexing solutions.
Set a single line in your config and make your handlers multiple times faster without changing a single line of code:
preload_handlers: true
For power users - This is like Loaders, but for all handlers by default.
Read more about Preload Optimization in the dedicated guide.
A highly requested feature contributed by one of our users @rori4
Specify a later start block for a contract to optimize indexing performance:
name: nft-indexer
description: NFT Factory
networks:
- id: 1337
start_block: 0
contracts:
- name: NftFactory
address: 0x4675a6B115329294e0518A2B7cC12B70987895C4
handler: src/EventHandlers.ts
events:
- event: SimpleNftCreated(string name, string symbol, uint256 maxSupply, address contractAddress)
- name: Nft
# No address field - we'll discover these addresses from SimpleNftCreated events
start_block:
handler: src/EventHandlers.ts
start_block: 30000000 # Overwrite the network start block
events:
- event: Transfer(address from, address to, uint256 tokenId)
In the Factory Contract example above, it'll register all NFT addresses, but it'll start processing Transfers only from the block 30000000.
To support new contributors, we have updated our CONTRIBUTING.md file with a detailed guide on navigating the HyperIndex codebase, as well as examples of what a change would look like. To make it even easier, we added .cursor rules to help develop new HyperIndex features.
All new projects are now created with an initial .cursor rules included to help you build indexers with the agent support.
Also, feel free to create PRs with the rules suggestions to improve the experience for everyone
import { experimental_createEffect, S } from "envio";
export const getMetadata = experimental_createEffect(
{
name: "getMetadata",
input: S.string,
output: {
description: S.string,
value: S.bigint,
},
cache: true, // Simply set cache to true
},
async ({ input, context }) => {}
})
Read more in the docs, learn how to persist the cache on reruns, and share it with Hosted Service (alpha).
ENVIO_HASURA=false environment variable.ENVIO_INDEXER_PORT env var (Instead of deprecated METRICS_PORT)ENVIO_PG_PASSWORD env var (Instead of deprecated ENVIO_POSTGRES_PASSWORD)context.Entity.getOrCreate and context.Entity.getOrThrow API// Before:
// let pool = await context.Pool.get(poolId);
// if (!pool) {
// pool = {
// id: poolId,
// totalValueLockedETH: 0n
// }
// context.Pool.set(pool);
// }
const pool = await context.Pool.getOrCreate({
id: poolId,
totalValueLockedETH: 0n
})
// Before:
// const pool = await context.Pool.get(poolId);
// if (!pool) {
// throw new Error(`Pool with ID ${poolId} is expected.`)
// }
const pool = await context.Pool.getOrThrow(poolId)
// Will throw: Entity 'Pool' with ID '...' is expected to exist.
// Or you can pass a custom message as a second argument:
const pool = await context.Pool.getOrThrow(poolId, `Pool with ID ${poolId} is expected.`)
These are additional helpers for DX improvements. You can access them from both handlers and loaders.
Loaders are a powerful feature to optimise indexer performance (Docs). The loader function is run twice: the first time in parallel for all events in the batch, and the second time immediately before the handler execution, to obtain up-to-date data. This part is unchanged, but we've done several improvements here:
Previously, even with Unordered Multichain Mode, we chose events to be batched from all available chains in the order of events on-chain. This is not bad, but for significant indexers relying on loader optimisation, it's better to have all events for a single chain in the batch. This way, there is a higher chance of achieving deduplication or batching optimization.
This is what we've done in the version. Now, we try to create a processing batch using events from one chain as much as possible, and then rotate to another chain for another batch.
To make HyperIndex run correctly, we used to have several requirements for the user's HyperIndex project, which was confusing and limited flexibility:
pnpm-workspaces.yaml file.npmrc file with shamefully hoisting dependenciesstart script in your package.json with ts-node generated/src/Index.bs.jsNow, none of this is necessary, and you're free to remove them. You can also remove ts-node from dependencies, just replace ts-node generated/src/Index.bs.js with envio start.
Unfortunately, it required changing .bs.js suffix to .res.js in our internal code. So if you, for some reason, relied on some internal fiels with .bs.js extention, you need to change them to .res.js manually. Also, for ReScript users, it's now required to configure .res.js suffix in your rescript.json file.
Note: Aggregate Queries might have a severe hit on Database performance. We generally recommend aggregating data using entities during indexing.
Now it's possible to enable Hasura Aggregate Queries using the ENVIO_HASURA_PUBLIC_AGGREGATE environment variable. You can set environment variables on our Hosted Service on Production plans.
ENVIO_HASURA_PUBLIC_AGGREGATE=["YourEntityName"]
Json field the second time #565loaderArgs and handlerArgs per event #540
Starting from version 2.21, you can use async contract registration.
This is a unique feature of Envio that allows you to perform an external call to determine the address of the contract to register.
NftFactory.SimpleNftCreated.contractRegister(async ({ event, context }) => {
const version = await getContractVersion(event.params.contractAddress);
if (version === "v2") {
context.addSimpleNftV2(event.params.contractAddress);
} else {
context.addSimpleNft(event.params.contractAddress);
}
});
accessList and EIP7702 authorizationList fields in EVM transactionJson type support for entity fields in graphql.schema fileExample:
type User {
id: ID!
greetings: [String!]!
latestGreeting: String!
numberOfGreetings: Int!
+ metadata: Json!
}
The new Effect API is a convenient way to perform external calls from your handlers. It's especially powerful when used with loaders:
It allows you to parallelize thousands of external calls from the processing batch, instead of executing them one by one.
Read more in the dedicated Loaders guide.
We made indexing with contract registrations much faster. It got so fast in the default mode that we disabled the preRegisterDynamicContracts option.
For the RPC data source, when we perform requests for event logs, different RPC providers might have different errors, with suggestions to use a different block range for retry. In the release, we added support for 9 more RPC providers to improve our RPC sync retry logic.
Fix the
--single-contractand--all-eventsflags for contract import from explorer #514
We started experimenting with Envio MCP, and creating an indexer using a CLI is definitely a fantastic feature.
Demo in Loom
<img width="1728" alt="Screenshot 2025-04-25 at 16 50 04" src="https://github.com/user-attachments/assets/f5b4bc99-08e5-46b1-8cda-b0deb2bf730a" />Track the indexer's progress and access the GraphQL Playground at the Development Console, with more features coming!
<img width="920" alt="image" src="https://github.com/user-attachments/assets/a805fd91-af97-4550-81bf-f497b2d73869" />Added the ability to pass params on a log call:
context.log.info("Sucessfully handled Transfer()", { from: event.params.from, to: event.params.to })
These params will be displayed in the logs in your terminal as well as Hosted Service.
You can also pass an error:
} catch (error) {
context.log.error("Failed ipfs call", error)
}
Now the eventFilters option can also accept a callback, allowing for building different filters depending on chainId:
import { ERC20 } from "generated";
const ZERO_ADDRESS = "0x0000000000000000000000000000000000000000";
const WHITELISTED_ADDRESSES = {
1: [
"0xf39Fd6e51aad88F6F4ce6aB8827279cffFb92266",
"0x70997970C51812dc3A010C7d01b50e0d17dc79C8",
],
100: ["0x3C44CdDdB6a900fa2b585dd299e03d12FA4293BC"],
};
ERC20.Transfer.handler(
async ({ event, context }) => {
//... your handler logic
},
{
wildcard: true,
eventFilters: ({ chainId }) => [
{ from: ZERO_ADDRESS, to: WHITELISTED_ADDRESSES[chainId] },
{ from: WHITELISTED_ADDRESSES[chainId], to: ZERO_ADDRESS },
],
}
);
chainId typeThe chainId type on event is now a union of chain ids the event belongs to. This is much safer than a number type used before.
You can now specify RPC providers for redundancy and failover, ensuring 100% uptime for your indexer. If HyperSync becomes unavailable, your indexer will automatically switch to an RPC provider.
To enable this, add an rpc field to your network configuration. It can be a URL or a list of RPC configuration objects:
networks:
- id: 137 # Polygon
start_block: 0
+ rpc: https://eth-mainnet.your-rpc-provider.com?API_KEY={ENVIO_MAINNET_API_KEY}
contracts:
- name: PolygonGreeter
address: 0x9D02A17dE4E68545d3a58D3a20BbBE0399E05c9c
The fallback RPC is triggered if the primary data source does not receive a new block within 20 seconds.
Adding an RPC provider is now as simple as setting a single URL:
networks:
- id: 137 # Polygon
start_block: 0
+ rpc: https://eth-mainnet.your-rpc-provider.com?API_KEY={ENVIO_MAINNET_API_KEY}
contracts:
- name: PolygonGreeter
address: 0x9D02A17dE4E68545d3a58D3a20BbBE0399E05c9c
Gain more control over your RPC setup by specifying it explicitly as a primary source:
networks:
- id: 137 # Polygon
start_block: 0
+ rpc:
+ url: https://eth-mainnet.your-rpc-provider.com?API_KEY={ENVIO_MAINNET_API_KEY}
+ for: sync
contracts:
- name: PolygonGreeter
address: 0x9D02A17dE4E68545d3a58D3a20BbBE0399E05c9c
Or define multiple RPC endpoints with custom settings:
networks:
- id: 137 # Polygon
start_block: 0
+ rpc:
+ - url: https://eth-mainnet.your-rpc-provider.com?API_KEY={ENVIO_MAINNET_API_KEY}
+ for: fallback
+ - url: https://eth-mainnet.your-free-rpc-provider.com
+ for: fallback
+ initial_block_interval: 1000
contracts:
- name: PolygonGreeter
address: 0x9D02A17dE4E68545d3a58D3a20BbBE0399E05c9c
unordered_multichain_mode to true for new projects. #462 (@manusoman)README.md. #461 (@DZakh)ENVIO_PG_PUBLIC_SCHEMA environment variable #445, #457
getWhere operatorYou can specify field selection for individual events. This feature is useful for optimizing RPC and HyperSync calls by fetching only the data relevant to specific events, avoiding over-fetching.
Per-event field selection overrides any global field selection, ensuring precise data handling for each event.
events:
- event: "Transfer(address indexed from, address indexed to, uint256 value)"
field_selection:
transaction_fields:
- "to"
- "from"
from, to, gasPrice, maxPriorityFeePerGas, maxFeePerGas, contractAddress transaction fields with the RPC data-source #371The Envio config file can now use Environment Variables to offer more flexibility. If you want to switch between different configurations quickly, you don't need to edit the config file each time; you can just set Environment Variables at run time.
For the Hosted Service users, it's now possible to set custom Environment Variables
Below is a simple example:
networks:
- id: ${ENVIO_CHAIN_ID:-137}
start_block: ${ENVIO_START_BLOCK:-45336336}
contracts:
- name: Greeter
address: ${ENVIO_GREETER_ADDRESSES}
Then you can run ENVIO_GREETER_ADDRESSES=0x9D02A17dE4E68545d3a58D3a20BbBE0399E05c9c pnpm dev, or set the values via the .env or the indexer settings page on the Hosted Service.
For the interpolation to be applied, the Environment Variable name should be placed in braces after a dollar sign (${VAR}).
For braced expressions, the following formats are supported:
Direct substitution
${VAR} -> value of VARDefault value
${VAR:-default} -> value of VAR if set and non-empty, otherwise default${VAR-default} -> value of VAR if set, otherwise defaultSchema, Belt, EntityHistory, InternalEntity, Entity, Utils #353This slims down the amount of data saved significantly for historical entity updates and improves db performance for large scale indexers.
You can now add "preRegisterDynamicContracts" flag to your event config. For all the events that have this config, there will be an end to end indexer run just for these events to run all the relevant contractRegister functions and collect all the dynamic contract addresses. The indexer then restarts with all these addresses from the start block configured for the network (rather than from the block that the contract gets registered). For a standard factory contract setup this drastically reduces indexing time since it doesn't result in many small block range queries but rather larger grouped queries.
PoolFactory.CreatePool.contractRegister(
({ event, context }) => {
context.addPool(event.params.pool);
},
{ preRegisterDynamicContracts: true },
);
Previously, you could only index LOG_DATA receipts. The release supports indexing MINT, BURN, TRANSFER, TRANSFER_OUT, and CALL receipt types.
Read more on the Fuel documentation page.
BigInt and BigDecimal fieldsWhen working with large integers or high-precision decimal numbers in your application, you might need to customize the precision and scale of your BigInt and BigDecimal fields. This ensures that your database stores these numbers accurately according to your specific requirements. If you know your numbers will not be too big, you can also optimize the database by not over-allocating on the precision.
Example:
type Product {
id: ID!
price: BigDecimal @numeric(precision: 10, scale: 2)
}
In this example, the price field can store numbers with up to 10 digits in total, with 2 digits after the decimal point (e.g., 99999999.99).
Read more on the Schema documentation page.
to_postgres_type function from codegen #224Index the whole chain by event signature without specifying exact contract addresses.
Here's an example of indexing all ERC20 Transfer events. No need to change the config file. Simply add the wildcard: true option to the second argument in your handler:
ERC20.Transfer.handler(
async ({ event, context }) => {},
{
wildcard: true,
}
)
Final touches implemented #171, #197
Filter events by indexed parameters. Simplify your handler's logic or get new indexing possibilities when joined with the Wildcard Indexing feature:
UniswapV3Factory.PoolCreated.handler(async ({ event, context }) => {},
{
wildcard: true,
eventFilters: [{ token0: DAI_ADDRESS }, { token1: DAI_ADDRESS }],
},
);
Both Wildcard Indexing and Event Filters are currently only supported with HyperSync. We are working on adding support for RPC mode in the following versions.
Implemented
The fuel ecosystem indexer is integrated into the main repository code. This was a huge refactoring which came with a big list of benefits and a big list of opportunities.
If you use envio@x.x.x-fuel version, you can switch to envio@2.3.0 and get:
envio@x.x.x-fuelraw_events configurationTalking about opportunities, this unblocks:
Mint/Burn/Call/TransferIn/TransferOut eventsImplemented #186, #190, #192, #193, #194, #196, #202
New Timestamp scalar type available in schema.graphql. It's represented as an instance of Date, making your handlers and queries more flexible, fast, and explicit.
Viem version to the latest V2 to fix type conflicts with the Viem version on user-side #159Be careful if you're using
ViemV1 in your indexer handlers. This change might cause unexpected behavior.
tsc --build to fail. Also, prevent this from happening in the future by adding a TypeScript type check to CI pipeline #165Ethers.Interface #155 - event: Assigned(address indexed recipientId, uint256 amount, address token)
- event: Assigned(address indexed recipientId, uint256 amount, address token, address sender)
name: AssignedWithSender
envio@2.1.5-fuel version which fixes dynamic contracts #126
