PNG uploads as a file attachment rather than image on develop

[kittykat]

Steps to reproduce

1. In a room, attach a PNG

Outcome

What did you expect?

See image preview

What happened instead?

See file:

Operating system

No response

Browser information

Chromium 110.0.5481.177 (Official Build) Arch Linux (64-bit)

URL for webapp

develop.element.io

Application version

Element version: 5ae88eb-react-32aa18ff2e0e-js-b4cdc5a92397 Olm version: 3.2.12

Homeserver

matrix.org

Will you send logs?

No

[t3chguy]

@kittykat please send logs, this happens when your browser's canvas can't generate a thumbnail typically

[kittykat]

Room ID: !UlIlSOXSmCVQUaTbFh:matrix.org
Message ID: $tYcrlMrJW6kiz8djEGuvcsJZTybepoFzZkjbss_cZZ0

[t3chguy]

2023-03-07T13:54:24.431Z E Invalid .png file header
Error: Invalid .png file header
    at extractChunks (https://develop.element.io/bundles/b4fa679a5cbd84250cf6/vendors~init.js:178640:31)
    at https://develop.element.io/bundles/b4fa679a5cbd84250cf6/init.js:63467:50
    at async Promise.all (index 0)
    at async loadImageElement (https://develop.element.io/bundles/b4fa679a5cbd84250cf6/init.js:63477:19)
    at async infoForImageFile (https://develop.element.io/bundles/b4fa679a5cbd84250cf6/init.js:63511:24)
    at async ContentMessages_ContentMessages.sendContentToRoom (https://develop.element.io/bundles/b4fa679a5cbd84250cf6/init.js:63825:29)

@kittykat would you be able to share the problem file (internally on the rageshake)

[kittykat]

@t3chguy done! It's a screenshot from a Pixel 6a running Graphene OS. It uploads fine as a photo when I attach it from Element Android

[t3chguy]

The file is called .png but it actually is a JPEG file

t3chguy@Michael-t3chguy-MBP ~/Desktop> file 223445092-3dab0ff7-8527-4861-929f-7e8dc5fc53dc.png 
223445092-3dab0ff7-8527-4861-929f-7e8dc5fc53dc.png: JPEG image data, JFIF standard 1.01, aspect ratio, density 1x1, segment length 16, baseline, precision 8, 1080x2400, components 3

[kittykat]

🤔 Wonder what my phone is doing…

Should JPEGs not be previewed? Is it assuming file type based on extension? If it is, can we get it to check file type instead?

[t3chguy]

JPEG should be, but for PNGs we need to check if it is hidpi by parsing it, so if we get given a faulty PNG we have to assume the file is broken not just mis-named

Is it assuming file type based on extension?

It is asking the browser but some browsers do indeed naively assume by file extension (sometimes being completely wrong for container formats like mkv which can be audio-only or video)

It is using this https://developer.mozilla.org/en-US/docs/Web/API/File/type

Note: Based on the current implementation, browsers won't actually read the bytestream of a file to determine its media type. It is assumed based on the file extension; a PNG image file renamed to .txt would give "text/plain" and not "image/png". Moreover, file.type is generally reliable only for common file types like images, HTML documents, audio and video. Uncommon file extensions would return an empty string. Client configuration (for instance, the Windows Registry) may result in unexpected values even for common types. Developers are advised not to rely on this property as a sole validation scheme.

can we get it to check file type instead?

We would need to bundle large libraries to parse image/video files to handle detection ourselves

[t3chguy]

@kittykat matrix-org/matrix-react-sdk#10308 should allow the PNG parsing to fail, but this file is likely to be problematic either way as we'll be sending it with a mime type of image/png (as the browser tells us) and other devices (native/mobile) may look at that, pick the specific renderer and fail to render it entirely. Not sure what Synapse's thumbnailer will do either, that could fail also.