Implement distributed execution #139
Conversation
fpetkovski
force-pushed
the
distributed-exec
branch
from
7f62d49 to
9836a25
Compare
fpetkovski
force-pushed
the
distributed-exec
branch
from
9836a25 to
0fc9d26
Compare
| remote(http_requests_total), | ||
| remote(http_requests_total) | ||
| ) | ||
| )`, |
There was a problem hiding this comment.
I am wondering for this query, do we want to do distributed execution or not? To me this seems better to just execute locally without querying remote engines. Since we are querying raw series, remote engines are just stores in this case
There was a problem hiding this comment.
I also thought about this, and the reason I thought it's beneficial to distribute the query is because PromQL is done over a limited number of steps. So by distributing the query we would "downsample" on the fly. When querying longer ranges, the number of samples going to the central querier will be much smaller than if we directly fetch chunks from storage.
For example, the resolution for a 30d query is 10368s. So we will remove a significant amount of samples because remote engines will sample the data.
There was a problem hiding this comment.
But for remote engines, all the raw samples are queried into memory, right? The query time downsampling is only beneficial to the central querier. One step further I am wondering if it is better to just do this when retrieving series from stores. Like thanos-io/thanos#4857
There was a problem hiding this comment.
The way it is currently implemented we would get the benefit of downsampling even if there is no function involved so it should be much better.
There was a problem hiding this comment.
But for remote engines, all the raw samples are queried into memory, right?
Yes, this is correct. However solving that at the storage level seems a bit more complicated and maybe something to tackle independently in the future. Maybe we could embed a VectorSelector operator in store components and use the same building blocks everywhere.
|
Should we merge this and continue iterating on main? The newly added code should be unused, so it won't affect anyone using the engine. |
This PR allows creating an instance of the engine capable of distributed execution.
The detailed proposal can be found here: thanos-io/thanos#6012.
Deduplication will be added as a follow up.