Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chunk fails if there's query scope #101

Closed
zoul0813 opened this issue Oct 23, 2017 · 1 comment
Closed

chunk fails if there's query scope #101

zoul0813 opened this issue Oct 23, 2017 · 1 comment

Comments

@zoul0813
Copy link
Contributor

chunk calls getAll in a loop, which is currently calling applyScopes. Since chunk does not create a new query with each loop, the previous scope functions are appended to the current query. With a basic model that has a single scope attached, chunk fails on the 301st call as FilterExpression has a limit of 300 expressions.

A "quick fix" for this is to add an additional parameter to getAll that determines whether or not to call applyScopes.

    protected function getAll($columns = [], $limit = -1, $use_iterator = true, $apply_scopes = true)
    {
        if($apply_scopes) $this->applyScopes();
        // ... trimmed
    }

Then modify chunk

    public function chunk($chunk_size, callable $callback)
    {
        $this->applyScopes();
        while (true) {
            $results = $this->getAll([], $chunk_size, false, false);

            call_user_func($callback, $results);

            if (empty($this->lastEvaluatedKey)) {
                break;
            }
        }
    }
@baopham
Copy link
Owner

baopham commented Oct 23, 2017

Could you update the title to: chunk fails if there's query scope

zoul0813 added a commit to zoul0813/laravel-dynamodb that referenced this issue Oct 23, 2017
…dly, eventually failing after 300 filter expressions have been applied.
zoul0813 added a commit to zoul0813/laravel-dynamodb that referenced this issue Oct 23, 2017
…y, stacking any model scopes until chunk eventually fails when >300 filter expressions exist on large data sets.
@zoul0813 zoul0813 changed the title chunk fails chunk fails if there's query scope Oct 23, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants