Getting Error when converting flux.dev with the latest version of deepcompressor #50
Comments
|
Here is my pip list btw: |
|
Hi @yibolu96, I recommend updating both the It would also be helpful if you could install Let me know what you find! |
|
/python3.11/site-packages/diffusers/models/embeddings.py", line 1208, in apply_rotary_emb I have the same problem and would like to ask how to solve it |
These two methods have no effect. Using dev/v0.1.0 will result in additional compile errors. |
File "/root/anaconda3/envs/deepcompressor/lib/python3.11/site-packages/diffusers/models/embeddings.py", line 1208, in apply_rotary_emb I also encountered such a problem. Has this problem been solved? |
|
I also ran into this issue while following the exact instructions in this readme (i.e. flux schnell quant): Note that there are some dependency issues since this repo does not pin dependencies to exact versions: That was the only difference from the I used this docker image: Here are the most recent logs leading up to the error: I've tested this multiple times on different H100 machines (from Runpod): NVIDIA driver version: 560.35.05, CUDA Version: 12.6 @synxlin would you be able to provide a bit more detail on how to do this:
|
|
I've tried to do some logging after following the steps here exactly: The logs below refer to running this command, to be clear: In def apply_rotary_emb(
x: torch.Tensor,
freqs_cis: Union[torch.Tensor, Tuple[torch.Tensor]],
use_real: bool = True,
use_real_unbind_dim: int = -1,
) -> Tuple[torch.Tensor, torch.Tensor]:
"""
Apply rotary embeddings to input tensors using the given frequency tensor. This function applies rotary embeddings
to the given query or key 'x' tensors using the provided frequency tensor 'freqs_cis'. The input tensors are
reshaped as complex numbers, and the frequency tensor is reshaped for broadcasting compatibility. The resulting
tensors contain rotary embeddings and are returned as real tensors.
Args:
x (`torch.Tensor`):
Query or key tensor to apply rotary embeddings. [B, H, S, D] xk (torch.Tensor): Key tensor to apply
freqs_cis (`Tuple[torch.Tensor]`): Precomputed frequency tensor for complex exponentials. ([S, D], [S, D],)
Returns:
Tuple[torch.Tensor, torch.Tensor]: Tuple of modified query tensor and key tensor with rotary embeddings.
"""
if use_real:
cos, sin = freqs_cis # [S, D]
cos = cos[None, None]
sin = sin[None, None]
cos, sin = cos.to(x.device), sin.to(x.device)
if use_real_unbind_dim == -1:
# Used for flux, cogvideox, hunyuan-dit
x_real, x_imag = x.reshape(*x.shape[:-1], -1, 2).unbind(-1) # [B, S, H, D//2]
x_rotated = torch.stack([-x_imag, x_real], dim=-1).flatten(3)
elif use_real_unbind_dim == -2:
# Used for Stable Audio
x_real, x_imag = x.reshape(*x.shape[:-1], 2, -1).unbind(-2) # [B, S, H, D//2]
x_rotated = torch.cat([-x_imag, x_real], dim=-1)
else:
raise ValueError(f"`use_real_unbind_dim={use_real_unbind_dim}` but should be -1 or -2.")
print("x.shape:", x.shape)
print("cos.shape:", cos.shape)
print("x_rotated.shape:", x_rotated.shape)
print("sin.shape:", sin.shape)
out = (x.float() * cos + x_rotated.float() * sin).to(x.dtype)
return out
else:
# used for lumina
x_rotated = torch.view_as_complex(x.float().reshape(*x.shape[:-1], -1, 2))
freqs_cis = freqs_cis.unsqueeze(2)
x_out = torch.view_as_real(x_rotated * freqs_cis).flatten(3)
return x_out.type_as(x)And the result is: I've then added logging in if image_rotary_emb is not None:
from .embeddings import apply_rotary_emb
print("query.shape", query.shape)
print("image_rotary_emb", image_rotary_emb)
query = apply_rotary_emb(query, image_rotary_emb)
key = apply_rotary_emb(key, image_rotary_emb)Which produces: And in print("hidden_states.shape", hidden_states.shape)
print("encoder_hidden_states.shape", encoder_hidden_states.shape)
print("temb.shape", temb.shape)
print("image_rotary_emb[0].shape", image_rotary_emb[0].shape)
print("image_rotary_emb[1].shape", image_rotary_emb[1].shape)
print("image_rotary_emb", image_rotary_emb)
print("joint_attention_kwargs", joint_attention_kwargs)
encoder_hidden_states, hidden_states = block(
hidden_states=hidden_states,
encoder_hidden_states=encoder_hidden_states,
temb=temb,
image_rotary_emb=image_rotary_emb,
joint_attention_kwargs=joint_attention_kwargs,
)Which produces: And also in print("txt_ids.shape:", txt_ids.shape)
print("img_ids.shape:", img_ids.shape)
ids = torch.cat((txt_ids, img_ids), dim=0)
print("ids.shape:", ids.shape)
image_rotary_emb = self.pos_embed(ids)
print("image_rotary_emb[0].shape:", image_rotary_emb[0].shape)
print("image_rotary_emb[1].shape:", image_rotary_emb[1].shape)Which outputs: |
|
@synxlin @lmxyy I'm not sure if you received notification for the above messages, but if so, apologies for the extra ping. As mentioned along with the logs above, deepcompressor is currently completely broken - even the official example doesn't work at all when followed exactly:
Are you please provide any hint on how to fix this? 🙏 |
Hi, I'm getting this error when converting flux.dev, how can I fix this? Big thanks ahead
Here is my full command:
I removed the eval part since I just want to do the convert
The text was updated successfully, but these errors were encountered: