Add camera support for Surface Pro 9#1867
Conversation
- Add support for OV5693 and OV13858 camera sensors - Patch Intel INT3472 driver for camera power management - Update IPU bridge for Surface Pro 9 compatibility
|
@qzed Here are my changes! I would like to thank you and all the other contributors that make this project possible! I really appreciate the hard work! |
|
@toorajtaraz OK so
Bit annoying that it's necessary but not a problem
Probably it was designed for platforms where the ACPI itself manages power on / off rather than the driver, so all that's needed is for pm_runtime_resume/suspend() to be called and the ACPI framework manages the rest.
The code you've added looks broadly fine; might not all be necessary. Do you know which other GPIO types discrete.c handles?
Is that definitely necessary for the camera to work? They could be just for a peripheral to the camera like a lens motor or flash.
If it works roll with it. In theory the link frequency needs to be high enough to support the data-rate that you need to support the resolutions you want to support with the number of lanes that you have, and then is supposed to be fine-tuned to avoid EM interference with the rest of the board. I've never noticed any adverse affect from just sticking with whatever the driver already supports, and changing it to a new value involves calculation settings for the sensor's PLL which can be tiresome. Tl;dr - ideally we'd figure out what value Windows uses and make sure the driver supports that, but sticking to 540MHz will be fine.
I can help get the changes tidied and into a format that's acceptable upstream, but you've done the work so they should go into linux with your name on them :) Next steps I think:
|
|
Would this also apply to the Surface Pro 8? |
It appears to be using the same cameras/drivers, though I'm not at all experienced enough to say what this means with regard this patch. |
|
What about the SP7? That's what I've got on hand. |
I got it working on a Surface Pro 8 for short, now it doesnt work anymore. Sensor ov5693 has no power anymore |
|
I am willing to test this on Fedora 43 with SP9 if someone tells me how to apply the patch.. |
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras
|
Added to our kernel repos in linux-surface/kernel@d205c7e. |
|
i just updated kernel to the latest release and didn't get any joy, only meaningful message in dmesg was |
|
i think i might need to wait for debian-6.17.13-2 to hit the repo as i appear to have 6.17.13-surface-1 still? |
|
@anthonysomerset Yeah, requires 6.17.13-2. Should be available in the repos soon (~15 minutes). |
|
This works for the SP8 back camera too! |
|
thanks i must have done something wierd... maybe when trying to install the "additional" ipu6 driver available via ubuntu or something - my system seems to think i have 32 cameras and none of them work via ffplay or cheese my dmesg is not giving any errors and it also states that it dectected 2 cameras during boot otherwise |
|
I had a lot of trouble getting it to work on my browser but the steps below made it so that it worked. This method made my camera work on Teams. Hopefully it helps. :) Ensure that you have installed the appropriate camera modules. See relevant wiki section for your distro-specific command. # Create the keys to sign v4l2loopback
cd ~/mok
openssl req -new -x509 -newkey rsa:2048 -keyout MOK.priv -outform DER -out MOK.der -days 36500 -subj "/CN=My Module Signing Key/" -nodes
sudo mokutil --import MOK.der
sudo reboot
# Sign the key with the password when prompted in the blue screen
# Build v4l2loopback from source if you hit issues with the one provided by the package manager
cd ~/dev/
git clone https://github.com/umlaeute/v4l2loopback.git
cd v4l2loopback
make clean
make
# This path may vary, this works for Fedora. You should search where that binary is in your distribution.
sudo /usr/src/kernels/$(uname -r)/scripts/sign-file sha256 ~/mok/MOK.priv ~/mok/MOK.der v4l2loopback.ko
sudo make install
sudo depmod -a
# Check which was the device that was created the latest, in my case it was video 32
ls -la /dev/video*
# Toggle the power of the front camera, otherwise you'll probably see black.
echo auto | sudo tee /sys/bus/i2c/devices/i2c-OVTI5693:00/power/control
echo on | sudo tee /sys/bus/i2c/devices/i2c-OVTI5693:00/power/control
# Create the camera
sudo modprobe v4l2loopback exclusive_caps=1 card_label="Virtual Camera" video_nr=32
# Should see "Video Capture" otherwise it won't work in the browser
v4l2-ctl -d /dev/video32 --all | grep -A 10 "Capabilities"
# just command below in another terminal, leave running (in tmux for instance)
gst-launch-1.0 libcamerasrc ! videoflip method=rotate-180 ! videoconvert ! video/x-raw,format=I420 ! v4l2sink device=/dev/video32
# test that it runs
ffplay /dev/video32
# Check on the browser if it works |
thanks for posting this work around :) Im trying to follow the steps but not 100% sure im getting the correct outputs. ive posted the outputs below for review. ` ++++++ To sign the module, you must set KBUILD_SIGN_KEY/KBUILD_SIGN_CERT to point to the signing key/certificate! ++++++ If your certificate requires a password, pass it via the KBUILD_SIGN_PIN env-var! rm -f *~ sp8:~/dev/v4l2loopback$ make ++++++ To sign the module, you must set KBUILD_SIGN_KEY/KBUILD_SIGN_CERT to point to the signing key/certificate! ++++++ If your certificate requires a password, pass it via the KBUILD_SIGN_PIN env-var! Building v4l2-loopback driver... p8:~/dev/v4l2loopback$ sudo /usr/src/kernels/$(uname -r)/scripts/sign-file sha256 ~/mok/MOK.priv ~/mok/MOK.der v4l2loopback.ko resolved the above issue below with different file pathsp8:~/dev/v4l2loopback$ sudo /usr/src/linux-headers-6.18.2-surface-1/scripts/sign-file sha256 ~/mok/MOK.priv ~/mok/MOK.der v4l2loopback.ko sp8:~/dev/v4l2loopback$ sudo make install ++++++ To sign the module, you must set KBUILD_SIGN_KEY/KBUILD_SIGN_CERT to point to the signing key/certificate! ++++++ If your certificate requires a password, pass it via the KBUILD_SIGN_PIN env-var! make -C /lib/modules/
SUCCESS (if you got 'SSL errors' above, you can safely ignore them) sp8:~/dev/v4l2loopback$ sudo depmod -a sp8: sp8:~/dev/v4l2loopback$ gst-launch-1.0 libcamerasrc ! videoflip method=rotate-180 ! videoconvert ! video/x-raw,format=I420 ! v4l2sink device=/dev/video63 ` i'm pretty sure the cert was accepted on reboot but i could be wrong, as i don't really know what i'm looking for with the output its hard to pick what is incorrect and whats acceptable - any feedback would be greatly appreciated :) ""Cheers |
Yeah, @velocity08, you're right, that path is apparently only for Fedora, different distros have different paths for that executable.
You might need to install the appropriate modules, check the relevant wiki section for the install commands. Hopefully that should work! Also, thanks for pointing out that issue. |
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras Link: https://github.com/linux-surface/linux-surface/tree/54cbb6c15e5fd226f8ff19af15e7c3dc3eb86f3d Signed-off-by: Mingcong Bai <jeffbai@aosc.io> [Xi Ruoyao: remove some code conflicting with the upstream change.] Signed-off-by: Xi Ruoyao <xry111@xry111.site>
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras Link: https://github.com/linux-surface/linux-surface/tree/54cbb6c15e5fd226f8ff19af15e7c3dc3eb86f3d Signed-off-by: Mingcong Bai <jeffbai@aosc.io> [Xi Ruoyao: remove some code conflicting with the upstream change.] Signed-off-by: Xi Ruoyao <xry111@xry111.site>
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras Link: https://github.com/linux-surface/linux-surface/tree/4c136ec928ce0b458562c7533852db67a8e0dc91 Signed-off-by: Mingcong Bai <jeffbai@aosc.io>
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras Link: https://github.com/linux-surface/linux-surface/tree/4c136ec928ce0b458562c7533852db67a8e0dc91 Signed-off-by: Mingcong Bai <jeffbai@aosc.io>
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras Link: https://github.com/linux-surface/linux-surface/tree/4c136ec928ce0b458562c7533852db67a8e0dc91 Signed-off-by: Mingcong Bai <jeffbai@aosc.io>
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras [ Mingcong Bai: Since commit 2a7b765 ("platform/x86: int3472: Handle GPIO type 0x10 (DOVDD)") took the 0x10 entry used by the original patch for Surface Pro 9's secondary power rail, change the latter to 0x11 to adapt to this change. Link: linux-surface/linux-surface#2057 Link: linux-surface/linux-surface#2079 ] Link: https://github.com/linux-surface/linux-surface/tree/4c136ec928ce0b458562c7533852db67a8e0dc91 Signed-off-by: Mingcong Bai <jeffbai@aosc.io>
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras [ Mingcong Bai: Since commit 2a7b765 ("platform/x86: int3472: Handle GPIO type 0x10 (DOVDD)") took the 0x10 entry used by the original patch for Surface Pro 9's secondary power rail, change the latter to 0x11 to adapt to this change. Link: linux-surface/linux-surface#2057 Link: linux-surface/linux-surface#2079 ] Link: https://github.com/linux-surface/linux-surface/tree/4c136ec928ce0b458562c7533852db67a8e0dc91 Signed-off-by: Mingcong Bai <jeffbai@aosc.io>
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras [ Mingcong Bai: Since commit 2a7b765 ("platform/x86: int3472: Handle GPIO type 0x10 (DOVDD)") took the 0x10 entry used by the original patch for Surface Pro 9's secondary power rail, change the latter to 0x11 to adapt to this change. Link: linux-surface/linux-surface#2057 Link: linux-surface/linux-surface#2079 ] Link: https://github.com/linux-surface/linux-surface/tree/4c136ec928ce0b458562c7533852db67a8e0dc91 Signed-off-by: Mingcong Bai <jeffbai@aosc.io>
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras [ Mingcong Bai: Since commit 2a7b765 ("platform/x86: int3472: Handle GPIO type 0x10 (DOVDD)") took the 0x10 entry used by the original patch for Surface Pro 9's secondary power rail, change the latter to 0x11 to adapt to this change. Link: linux-surface/linux-surface#2057 Link: linux-surface/linux-surface#2079 ] Link: https://github.com/linux-surface/linux-surface/tree/4c136ec928ce0b458562c7533852db67a8e0dc91 Signed-off-by: Mingcong Bai <jeffbai@aosc.io>
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras [ Mingcong Bai: Since commit 2a7b765 ("platform/x86: int3472: Handle GPIO type 0x10 (DOVDD)") took the 0x10 entry used by the original patch for Surface Pro 9's secondary power rail, change the latter to 0x11 to adapt to this change. Link: linux-surface/linux-surface#2057 Link: linux-surface/linux-surface#2079 ] Link: https://github.com/linux-surface/linux-surface/tree/4c136ec928ce0b458562c7533852db67a8e0dc91 Signed-off-by: Mingcong Bai <jeffbai@aosc.io>
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras [ Mingcong Bai: Since commit 2a7b765 ("platform/x86: int3472: Handle GPIO type 0x10 (DOVDD)") took the 0x10 entry used by the original patch for Surface Pro 9's secondary power rail, change the latter to 0x11 to adapt to this change. Link: linux-surface/linux-surface#2057 Link: linux-surface/linux-surface#2079 ] Link: https://github.com/linux-surface/linux-surface/tree/4c136ec928ce0b458562c7533852db67a8e0dc91 Signed-off-by: Mingcong Bai <jeffbai@aosc.io>
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras [ Mingcong Bai: Since commit 2a7b765 ("platform/x86: int3472: Handle GPIO type 0x10 (DOVDD)") took the 0x10 entry used by the original patch for Surface Pro 9's secondary power rail, change the latter to 0x11 to adapt to this change. Link: linux-surface/linux-surface#2057 Link: linux-surface/linux-surface#2079 ] Link: https://github.com/linux-surface/linux-surface/tree/4c136ec928ce0b458562c7533852db67a8e0dc91 Signed-off-by: Mingcong Bai <jeffbai@aosc.io>
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras [ Mingcong Bai: Since commit 2a7b765 ("platform/x86: int3472: Handle GPIO type 0x10 (DOVDD)") took the 0x10 entry used by the original patch for Surface Pro 9's secondary power rail, change the latter to 0x11 to adapt to this change. Link: linux-surface/linux-surface#2057 Link: linux-surface/linux-surface#2079 ] Link: https://github.com/linux-surface/linux-surface/tree/4c136ec928ce0b458562c7533852db67a8e0dc91 Signed-off-by: Mingcong Bai <jeffbai@aosc.io>
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras [ Mingcong Bai: Since commit 2a7b765 ("platform/x86: int3472: Handle GPIO type 0x10 (DOVDD)") took the 0x10 entry used by the original patch for Surface Pro 9's secondary power rail, change the latter to 0x11 to adapt to this change. Link: linux-surface/linux-surface#2057 Link: linux-surface/linux-surface#2079 ] Link: https://github.com/linux-surface/linux-surface/tree/4c136ec928ce0b458562c7533852db67a8e0dc91 Signed-off-by: Mingcong Bai <jeffbai@aosc.io>
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras [ Mingcong Bai: Since commit 2a7b765 ("platform/x86: int3472: Handle GPIO type 0x10 (DOVDD)") took the 0x10 entry used by the original patch for Surface Pro 9's secondary power rail, change the latter to 0x11 to adapt to this change. Link: linux-surface/linux-surface#2057 Link: linux-surface/linux-surface#2079 ] Link: https://github.com/linux-surface/linux-surface/tree/4c136ec928ce0b458562c7533852db67a8e0dc91 Signed-off-by: Mingcong Bai <jeffbai@aosc.io>
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras [ Mingcong Bai: Since commit 2a7b765 ("platform/x86: int3472: Handle GPIO type 0x10 (DOVDD)") took the 0x10 entry used by the original patch for Surface Pro 9's secondary power rail, change the latter to 0x11 to adapt to this change. Link: linux-surface/linux-surface#2057 Link: linux-surface/linux-surface#2079 ] Link: https://github.com/linux-surface/linux-surface/tree/4c136ec928ce0b458562c7533852db67a8e0dc91 Signed-off-by: Mingcong Bai <jeffbai@aosc.io>
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras [ Mingcong Bai: Since commit 2a7b765 ("platform/x86: int3472: Handle GPIO type 0x10 (DOVDD)") took the 0x10 entry used by the original patch for Surface Pro 9's secondary power rail, change the latter to 0x11 to adapt to this change. Link: linux-surface/linux-surface#2057 Link: linux-surface/linux-surface#2079 ] Link: https://github.com/linux-surface/linux-surface/tree/4c136ec928ce0b458562c7533852db67a8e0dc91 Signed-off-by: Mingcong Bai <jeffbai@aosc.io>
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras [ Mingcong Bai: Since commit 2a7b765 ("platform/x86: int3472: Handle GPIO type 0x10 (DOVDD)") took the 0x10 entry used by the original patch for Surface Pro 9's secondary power rail, change the latter to 0x11 to adapt to this change. Link: linux-surface/linux-surface#2057 Link: linux-surface/linux-surface#2079 ] Link: https://github.com/linux-surface/linux-surface/tree/4c136ec928ce0b458562c7533852db67a8e0dc91 Signed-off-by: Mingcong Bai <jeffbai@aosc.io>
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras [ Mingcong Bai: Since commit 2a7b765 ("platform/x86: int3472: Handle GPIO type 0x10 (DOVDD)") took the 0x10 entry used by the original patch for Surface Pro 9's secondary power rail, change the latter to 0x11 to adapt to this change. Link: linux-surface/linux-surface#2057 Link: linux-surface/linux-surface#2079 ] Link: https://github.com/linux-surface/linux-surface/tree/4c136ec928ce0b458562c7533852db67a8e0dc91 Signed-off-by: Mingcong Bai <jeffbai@aosc.io>
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras [ Mingcong Bai: Since commit 2a7b765 ("platform/x86: int3472: Handle GPIO type 0x10 (DOVDD)") took the 0x10 entry used by the original patch for Surface Pro 9's secondary power rail, change the latter to 0x11 to adapt to this change. Link: linux-surface/linux-surface#2057 Link: linux-surface/linux-surface#2079 ] Link: https://github.com/linux-surface/linux-surface/tree/4c136ec928ce0b458562c7533852db67a8e0dc91 Signed-off-by: Mingcong Bai <jeffbai@aosc.io>
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras [ Mingcong Bai: Since commit 2a7b765 ("platform/x86: int3472: Handle GPIO type 0x10 (DOVDD)") took the 0x10 entry used by the original patch for Surface Pro 9's secondary power rail, change the latter to 0x11 to adapt to this change. Link: linux-surface/linux-surface#2057 Link: linux-surface/linux-surface#2079 ] Link: https://github.com/linux-surface/linux-surface/tree/4c136ec928ce0b458562c7533852db67a8e0dc91 Signed-off-by: Mingcong Bai <jeffbai@aosc.io>
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras [ Mingcong Bai: Since commit 2a7b765 ("platform/x86: int3472: Handle GPIO type 0x10 (DOVDD)") took the 0x10 entry used by the original patch for Surface Pro 9's secondary power rail, change the latter to 0x11 to adapt to this change. Link: linux-surface/linux-surface#2057 Link: linux-surface/linux-surface#2079 ] Link: https://github.com/linux-surface/linux-surface/tree/4c136ec928ce0b458562c7533852db67a8e0dc91 Signed-off-by: Mingcong Bai <jeffbai@aosc.io>
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras [ Mingcong Bai: Since commit 2a7b765 ("platform/x86: int3472: Handle GPIO type 0x10 (DOVDD)") took the 0x10 entry used by the original patch for Surface Pro 9's secondary power rail, change the latter to 0x11 to adapt to this change. Link: linux-surface/linux-surface#2057 Link: linux-surface/linux-surface#2079 ] Link: https://github.com/linux-surface/linux-surface/tree/4c136ec928ce0b458562c7533852db67a8e0dc91 Signed-off-by: Mingcong Bai <jeffbai@aosc.io>
Experimental camera support for the Surface Pro 9. Link: linux-surface/linux-surface#1867 Patchset: cameras [ Mingcong Bai: Since commit 2a7b765 ("platform/x86: int3472: Handle GPIO type 0x10 (DOVDD)") took the 0x10 entry used by the original patch for Surface Pro 9's secondary power rail, change the latter to 0x11 to adapt to this change. Link: linux-surface/linux-surface#2057 Link: linux-surface/linux-surface#2079 ] Link: https://github.com/linux-surface/linux-surface/tree/4c136ec928ce0b458562c7533852db67a8e0dc91 Signed-off-by: Mingcong Bai <jeffbai@aosc.io>
I don't feel like the changes I've made are robust enough to be added to the kernel repo or even be merged in this repo for that matter. So this is just to start a thread with my patch so that the community can test and complete what I've done.
There are a ton of quirks that I cannot fully explain so here goes what I did:
ov5693.c: I just needed to addOVTI5693ACPI ID to make it bind properly!ov13858.c: I have no idea why, but this code basically did none of the important things. I'm not experienced enough to know if this was intentional or just incomplete code. So I added regulators "avdd" andpwr1, which is my best guess for the type 0x08, as the only thing that worked was when that pin was treated as a regulator pin. The name, of course, is made up by me. By looking at other driver codes, I added callbacks for suspend and resume to hopefully not waste energy. I'm not sure if that goal was achieved, but it seems to work. I also added code for GPIO reset control and external clock (xvclk) management.discrete.c (INT3472): Oh this one was a weird one. For some reason, unrecognized GPIO types would make the entire thing fail. I have no idea what 0x10 is supposed to be, but I just ended up skipping it instead of failing. As for 0x08, as mentioned before, I named it "pwr1" and treated it just like a regulator.ipu-bridge.c: I added OVTI5693 (front camera): 1 lane, 419.2 MHz link frequency and OVTID858 (rear camera): 4 lanes, 540 MHz link frequency. Honestly, the 4 x 540 MHz was a combination of guessing and looking at ov13858. Worked on my first try, so I didn't think twice about it. (I also skimmed through a weird 140-page document I found on the internet about the sensor)And now it's time for the weird stuff!
At first, I added udev rules that would do the equivalent of these commands:
When I did that, things would "work" but everything would look greenish! So I removed that rule and figured I'd run these commands manually, but surprise surprise, the privacy LED would turn on but nothing, just a black screen!! I had no idea what to do at that point, so I just echoed "auto" and a couple of "on"s back to back, and it started working. Once it started to work, it would just reliably work every time. I know, it's weird!
I would love to keep working on this until it's ready to use, but I'm a PhD candidate with very limited time. If someone's interested in picking up the rest of the work, I would happily provide all the information I have! I did mention this in an issue, but dumping ACPI tables and decompiling them proved to be invaluable.
Cheers everyone!
P.S.
These are a couple of notes about my machine and the commands I ran: