Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compile byte code during ImageSpec install #3175

Merged
merged 1 commit into from
Mar 5, 2025

Conversation

thomasjpfan
Copy link
Member

@thomasjpfan thomasjpfan commented Mar 5, 2025

Why are the changes needed?

This PR adds byte code compiling to the image spec, which reduces runtime at the cost of a slightly bigger image. With this simple workflow:

from flytekit import ImageSpec, task

image = ImageSpec(
    name="bytecode",
    registry="localhost:30000",
    packages=["torch"],
)

@task(container_image=image)
def hello() -> float:
    import torch

    x = torch.ones(10, dtype=torch.float32)
    return x.sum().item()

I got these runtimes:

new node image on node image size
compile 46s 20s 5.86GB
no-compile 1m5s 33s 5.72GB

What changes were proposed in this pull request?

Adds UV_COMPILE_BYTECODE=1 to the image builder.

How was this patch tested?

I ran the workflow above and benchmarked the times.

Summary by Bito

This PR introduces bytecode compilation to the ImageSpec installation process by adding the UV_COMPILE_BYTECODE flag in the image builder file. The modification optimizes image build configuration as shown by benchmarks, enhancing functionality while remaining straightforward and testable.

Unit tests added: False

Estimated effort to review (1-5, lower is better): 1

@flyte-bot
Copy link
Contributor

flyte-bot commented Mar 5, 2025

Code Review Agent Run #88d61b

Actionable Suggestions - 0
Review Details
  • Files reviewed - 1 · Commit Range: 19c35b1..19c35b1
    • flytekit/image_spec/default_builder.py
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful

AI Code Review powered by Bito Logo

@flyte-bot
Copy link
Contributor

Changelist by Bito

This pull request implements the following key changes.

Key Change Files Impacted
New Feature - Enable Bytecode Compilation

default_builder.py - Inserted the UV_COMPILE_BYTECODE=1 environment variable to enable byte code compilation during ImageSpec installation.

@eapolinario
Copy link
Collaborator

Is there any downside to having this always enabled?

@thomasjpfan
Copy link
Member Author

Is there any downside to having this always enabled?

Slightly longer build times and slight larger images. With just PyTorch, the image is 5.86GB with compile and 5.72GB without.

Note that normal pip install (without uv) compiles by default.

@eapolinario eapolinario enabled auto-merge (squash) March 5, 2025 15:49
@eapolinario eapolinario merged commit fd3941b into flyteorg:master Mar 5, 2025
112 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants