Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Estimate Gas Not Working With Geth #290

Closed
nlordell opened this issue Nov 29, 2019 · 1 comment · Fixed by #291
Closed

Estimate Gas Not Working With Geth #290

nlordell opened this issue Nov 29, 2019 · 1 comment · Fixed by #291

Comments

@nlordell
Copy link
Contributor

nlordell commented Nov 29, 2019

Eth::estimate_gas does not seem to work when connected to a geth node. From what I can gather this is because ethereum/go-ethereum#2586 where geth does not implement the documented Web3 specification for eth_estimateGas.

According to the specification, eth_estimateGas expects the same parameters as eth_call, that is (in rust-web3 terms) a CallRequest and a BlockNumber. However, geth seems to only accept a CallRequest and with fail when a BlockNumber is specified.

This is a problem when using any geth node (such as when using Infura). The following code illustrates this problem:

use futures::future::Future;
use serde_json::json;
use std::env;
use web3::api::Web3;
use web3::transports::Http;
use web3::types::CallRequest;
use web3::Transport;

fn main() {
    let (eloop, http) = {
        let project_id = env::var("INFURA_PROJECT_ID").unwrap();
        Http::new(&format!("https://rinkeby.infura.io/v3/{}", project_id)).unwrap()
    };
    eloop.into_remote();
    let web3 = Web3::new(http.clone());

    println!(
        "Client version: {:?}",
        web3.web3().client_version().wait().unwrap()
    ); // Client version: "Geth/v1.9.7-omnibus-cc8122b7-20191122/linux-amd64/go1.13.4"

    let gas = web3
        .eth()
        .estimate_gas(
            CallRequest {
                from: None,
                to: "000102030405060708090a0b0c0d0e0f10111213".parse().unwrap(),
                gas: None,
                gas_price: None,
                value: None,
                data: None,
            },
            None,
        )
        .wait();
    println!("{:?}", gas); // Err("too many arguments, want at most 1")

    let gas = http
        .execute(
            "eth_estimateGas",
            vec![json!({
                "to": "0x000102030405060708090a0b0c0d0e0f10111213",
            })],
        )
        .wait();
    println!("{:?}", gas); // Ok("0x5208") == 21000 == BASE_GAS_PRICE (as expected)
}

I suggest doing as web3.py does (ethereum/web3.py#1046) and only send the CallRequest parameter on a eth_estimateGas request when the specified BlockNumber is None. This works with other nodes such as parity and development nodes such as Ganache so it shouldn't impact rust-web3 users while allowing them to successfully call eth_estimateGas when connected to geth nodes.

This feels icky as it requires rust-web3 to work around an issue that is arguably a geth bug, but seeing as ethereum/go-ethereum#2586 has been open since 2016 I think it is a worthwhile workaround (especially since geth is widely used, and used with Infura).

@tomusdrw
Copy link
Owner

tomusdrw commented Dec 1, 2019

Yeah, not sending the second parameter in case BlockNumber is None seems like a good solution to me. We already have some node-specific workarounds in the codebase afair, so I'm happy to accept a PR fixing this.

nlordell added a commit to cowprotocol/ethcontract-rs that referenced this issue Dec 4, 2019
This implements a workaround to tomusdrw/rust-web3#290. While the issue has been fixed on master, it has not been released to crates.io.

Fixes #47 

### Test Plan

CI
MartinquaXD added a commit to cowprotocol/services that referenced this issue Jan 8, 2025
…3220)

# Description
For a while we had mysteriously failing transactions. What I mean is
that we execute `eth_estimateGas` to see if a transaction would still
work. If it fails we log all the data needed to re-simulate it on
tenderly. But when you then actually re-simulate on tenderly the
transaction would work.

The solution unravels with this
[issue](tomusdrw/rust-web3#290).
The spec for `eth_estimateGas` mentions 2 arguments: the tx and an
identifier for a block. The block is supposed to be optional but geth
had a bug for a very long time. When you don't pass a block argument it
should be sent over the wire as `null`. But geth tried to deserialize
`null` which then caused these calls to fail.

As a workaround the `web3` crate stopped serializing the block tag if it
was `None`. Eventually geth fixed this issue and implemented it such
that a missing block tag would default to `pending`.
Also a rule of thumb in node development is "if in doubt do whatever
geth does" so reth also implemented the default to `pending`.

Defaulting to `pending` is what can make these reverts unpredictable.
When you simulate the tx on `pending` the node takes the state of the
`latest` block PLUS transactions that it currently has in its mempool.
So whenever we log a reverting tx that works on tenderly it only failed
in our node because some tx in the nodes mempool interfered with the
solution.
Interestingly there also a lot of cases where you can even simulate the
tx on the next block. In those cases simulating on `pending` caused the
driver completely erroneously since the problematic tx didn't even make
it into the next block.

<details>
<summary>example</summary>


[log](https://aws-es.cow.fi/_dashboards/app/discover#/doc/86e4a5a0-4e4b-11ef-85c5-3946a99ed1a7/cowlogs-prod-2025.01.06?id=1F1ROpQBOTpjfSxxfB9z)
```
2025-01-06T06:33:25.748Z  INFO request{id="4114843"}:/solve{solver=extzeroex-solve auction_id=9990122}: driver::infra::observe: discarded solution: settlement encoding id=Id { id: 17739308, merged_solutions: [22] } err=Simulation(Revert(RevertError { err: Blockchain(AccessList(String("execution reverted"))), tx: Tx { from: Address(0x28b1bd44996105b5c14c4de41093226ff78a4eb1), to: Address(0x9008d19f58aabd9ed0d60971565aa8510560ab41), value: Ether(0), input: 0x13d79a0b0000000000000000000000000000000000000000000000000000000000000080000000000000000000000000000000000000000000000000000000000000012000000000000000000000000000000000000000000000000000000000000001c000000000000000000000000000000000000000000000000000000000000003e00000000000000000000000000000000000000000000000000000000000000004000000000000000000000000839d4a05d6d23a22365bd90c2114c199d53e674c000000000000000000000000eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee000000000000000000000000839d4a05d6d23a22365bd90c2114c199d53e674c000000000000000000000000eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee0000000000000000000000000000000000000000000000000000000000000004000000000000000000000000000000000000000000000000017c836d081141b7000000000000000000000000000000000000000000000034cafbfac2c90f727a000000000000000000000000000000000000000000000000017c836d081141b700000000000000000000000000000000000000000000003635c9adc5dea0000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000030000000000000000000000006a7fe93d1e194d31b8076c255779c4b12a45192c00000000000000000000000000000000000000000000003635c9adc5dea00000000000000000000000000000000000000000000000000000010df3f85654f86700000000000000000000000000000000000000000000000000000000677b7e7d5b203ebd4ff6f6667daa775bd3de4410b0b834f059fccecd85660cb72bf031e30000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000003635c9adc5dea00000000000000000000000000000000000000000000000000000000000000000016000000000000000000000000000000000000000000000000000000000000000410fd3b1f82fc78a190ccf38bbfa08652ca518ef025802fe671c0b7e1b8cc1c8e762470c92b313dd597fde1022168575929770076f30cc6581377d174f76ce675e1c0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000600000000000000000000000000000000000000000000000000000000000000080000000000000000000000000000000000000000000000000000000000000046000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000003000000000000000000000000000000000000000000000000000000000000006000000000000000000000000000000000000000000000000000000000000001400000000000000000000000000000000000000000000000000000000000000300000000000000000000000000839d4a05d6d23a22365bd90c2114c199d53e674c000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000600000000000000000000000000000000000000000000000000000000000000044095ea7b3000000000000000000000000def1c0ded9bec7f1a1670819833240f027b25eff00000000000000000000000000000000000000000000003635c9adc5dea0000000000000000000000000000000000000000000000000000000000000000000000000000000000000def1c0ded9bec7f1a1670819833240f027b25eff000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000600000000000000000000000000000000000000000000000000000000000000128d9627aa4000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000003635c9adc5dea000000000000000000000000000000000000000000000000000000186565d4040733b00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000839d4a05d6d23a22365bd90c2114c199d53e674c000000000000000000000000c02aaa39b223fe8d0a0e5c4f27ead9083c756cc2869584cd0000000000000000000000009008d19f58aabd9ed0d60971565aa8510560ab410000000000000000000000000000000000000000b507dba868e8950578d34d3f000000000000000000000000000000000000000000000000000000000000000000000000c02aaa39b223fe8d0a0e5c4f27ead9083c756cc20000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000006000000000000000000000000000000000000000000000000000000000000000242e1a7d4d000000000000000000000000000000000000000000000000017c836d081141b70000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000986fea, access_list: AccessList({}) }, block: BlockNo(21563660) }))
```

Results in this
[simulation](https://dashboard.tenderly.co/cow-protocol/staging/simulator/0b89b4d0-418f-40e3-8502-59c85e4da885)
that even works 2 blocks past the block we logged.
</details>

# Changes
Always estimate gas and create access lists on the `latest` block
instead of `pending`.
This should be the better default for 2 reasons:
1. it makes our simulation reverts consistent with what we see on
tenderly
2. most blocks get built by sophisticated algorithms including and
arranging blocks in a way that maximizes MEV. That means whatever
transactions ordering the node applies in the `pending` likely isn't
accurate anyway.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants