Little update - advanced FPS control (0.2 FPS possible)

Hi,

You've probably noticed that I've been a little slow during the last few weeks. This was due to my studies - I've got a PhD degree in engineering :)

I'm very happy to see the progress from other developers, for example, Sztupy with Android ML controller and Lua scripting, Coutts with the initial work in porting ML on 5D classic, and of course, g3gg0 with the new FPS discoveries and firmware analysis.

Now I've been trying to get back to speed, so here's another experimental update.

The main feature is full control over FPS, which was researched here: http://groups.google.com/group/ml-devel/browse_thread/thread/b1dcd0301fe35b44

In a nutshell: we can now record at very low FPS (lower limit being close to 0.2 FPS); also, we can get exact FPS values (24.000 fps) or even extreme jello effects.

Details:

There are two degrees of freedom for obtaining a given FPS (we can change two timer registers). Besides FPS changing, each register has some interesting side effects: one of them alters the amount of jello effects (rolling shutter), the other one increases exposure time (converging to 1/fps at very low frame rates). So, to get some fixed FPS value, there are a lot of solutions; we may choose one according to some other criteria, for example:

- 360 degree shutter speed at very low FPS (we already have this in previous implementation)
- Exact FPS values (24.000, or 30.000 on 5D2)
- Extreme rolling shutter (try it at 2.5fps)

Other things worth trying:
- On 500D, the first timer register has a very high value compared to other cameras. We might have a nice surprise.
- Try measuring rolling shutter. It might be possible to reduce the jello effect by a tiny amount (roughly 5% or 10% or improvement in theory)

Caveats:
- On 60D/600D I've removed some features which were working only on those cameras: that is, shutter speed linked to FPS, and extremely fast shutter speeds (faster than 1/8000). The math behind FPS is getting complex, so maintaining two separate versions would mean a lot of work and would be very error-prone.
- I've removed the slit-scan feature (it's now replaced by high jello mode in FPS override).

--------------------------

Not related to FPS:

Stability improvements:
- Two new stability tests - these uncovered a bug in the menu backend which was present in all ML versions that had a menu. This bug caused problems even if all ML features were disabled (only a blank menu box was enough). It was caused by a Canon API call which was not fully understood. I've never noticed that bug in practice (only in automatic tests).
- Menu backend was rewritten using the new dialog box API (and, of course, passes all current tests).
- Crash logging: ML intercepts ERR70 (maybe other errors too) and saves a log file which contains the error message from Canon, and also ML version info. If anything goes wrong, these log files should help me isolate the problem.

Other important fixes:
- Bracketing (fixed issue 1083).
- HDR video sync fixed in 5D2.
- Bulb ramping might work in 600D (not tested, but a 550D user had the same problem).

Other enhancements:
- Help for menu navigation keys (should answer all questions like "how to change cropmarks / intervalometer / bracketing settings / etc").
- For items with many values (for example, audio gain), use SET + Left/Right, rather than SET/PLAY. Hopefully a bit more intuitive.
- UniWB support (in the white balance submenu).
- For 50D, open ML menu with the Delete button - like in all other cameras.
- Highlight++ was moved under ISO submenu (renamed to Clipping Point). This is still under research. I've added a function (ISO submenu: Maximize dynamic range), which will auto-detect the optimal clipping point (that is, it lowers digital gain as much as possible, while keeping white at 255). My preliminary advice is to set a full-stop ISO (100, 200... 1600) and then run ML routine for optimizing digital gain.

Sztupy's extensions:
- PTP support for ML controller is enabled, but not tested. I did not change anything here.
- Plugin support is enabled, but fully optional. If you copy the extra files from Sztupy's zip ( http://groups.google.com/group/ml-devel/browse_thread/thread/7bfc8b67a1748bcd ), you will have Lua scripting. If not... ML will not complain.

Full changelog: https://bitbucket.org/hudson/magic-lantern/changesets

With 5D2 I still have a problem with incomplete shutdowns. It doesn't happen often, but when it does, it would drain the battery completely. I didn't notice it on the other cameras, and I couldn't find any pattern that might cause this. Any hint on reproducing this problem is welcome.

Administrative: I've disabled the mirroring of ML issues on the group. You can still monitor them here:
https://bitbucket.org/hudson/magic-lantern/issues

Cleanup: memory usage starts to increase (a bit too much for my taste), so I'm listening to your suggestions - what features can be removed to keep code size down? Maybe they can be refactored as Lua scripts or plugins.

This autoexec should run on 550D 1.0.9, 60D 1.0.1, 600D 1.0.1, 50D 1.0.8, 500D 1.1.1 and 5D2 2.1.1. That is, it still runs on the old firmware versions. The source code is prepared for latest Canon upgrades (if you compile from source, you will get an autoexec for 50D 1.0.9, 5D2 2.1.2 and 600D 1.0.2), but I prefer to post binaries for the old firmware versions until I'll be ready for the next major release. This is to make the upgrade process less confusing for users.

I did not do any extensive testing, so expect rough edges. But as a proof of concept it should be okay.

That's it for today. Thanks to everybody for not losing the faith, for keeping the development and the discussions alive, and for your continued support!

P.S. Xaos was faster than me and already recorded a few timelapses at
various frame rates:

http://www.youtube.com/watch?v=VgFv5uo0geQ - 0.2fps
http://www.youtube.com/watch?v=NHdfrnea6bs - 0.5fps
http://www.youtube.com/watch?v=wpy1hIjqbKE - 1fps
http://www.youtube.com/watch?v=hOCtjAKbBns - 5fps

from http://www.youtube.com/user/hipotures
> The main feature is full control over FPS, which was researched here:http://groups.google.com/group/ml-devel/browse_thread/thread/b1dcd030...
> files from Sztupy's zip (http://groups.google.com/group/ml-devel/browse_thread/thread/7bfc8b67...),
>  autoexec.bin
> 2253KViewDownload

https://bitbucket.org/hudson/magic-lantern/changesets - my hottest rss :D

> --
> http://magiclantern.wikia.com/
>
> To post to this group, send email to ml-d...@googlegroups.com
> To unsubscribe from this group, send email to ml-devel+u...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/ml-devel?hl=en

2012.03.23. 22:50 keltez�ssel, Alex �rta:I'm still doing some work on the lua/plugin/PTP system. After that I
think some of the more "obscure" functionality could be easily split to
be plugins with close to zero performance drop.

SztupY

yay!
- Jakob

I'm working on a video with low fps (and CBR=3x ), 3-4 hours recording
(RT) without interrupting (single file) :D Timer at right-top corner
is a little out of control :)

> ml-devel+u...@googlegroups.com

Do we have to address you as Doctor Alex now? Congratulating Alex!!!

On my 5d Mkii, when trying the fps override -in rec standby mode, if i zoom in on live view it goes crazy and distorted until i return to normal zoom level.

More info: it only happens on Low Jello and Exact fps mode.

Hello,

when I try to shoot a video on my 550D with 0.2fps the camera crashes
after 20sec and displays err 80.
If I shoot with 0.5 fps this won't happen.

If you have trouble recording, try disabling FPS override, then start
recording, then re-enable FPS override again. This solves the problem
at 0.2fps on 550D.

Congratulations, doctor Alex !!!!

It's been awhile since I commented, I just wanted to say this new
development with extremely low FPS settings is incredible! I'm making
a short film right now and I'm going to test this on a scene I wanted
to shoot in time lapse!

Also, congrats on the engineering degree!

Thanks, that works.
Is 0.1fps or even lower possible on the 550D? Because this is getting
excited for Astro-timelapses because it does not rise the shutter
count.

Congrats on your Ph.d Alex!

Wow nice feature. Did some preliminary testing on my 500D. The new menu navigation & FPS settings work & are very nice. But the video is washed out(no colors) & distorted. Its hard to understand whats on screen. But its proof that it does work, that too for a prelimnary version. Gonna test it a bit more.  Would you like a sample video? 

Missed you man! Welcome back! Congrats on your Phd degree!
//D
> <veeraragava...@gmail.com>wrote:

Hi Alex -- congrats mate on passing the exams mate. It interesting you
mention about the power drain on the 5D2. I thought it was my
imagination, but lately i have noticed that random times when i go to
use my camera, the batteries were totally flat, even although i know
they were fully charged the previous day, and i only used the camera
perhaps for about 10 or 20 pictures that day. I put it down to that
maybe i had not turned the camera off properly or (i use a grip with 2
batteries) that maybe one of my cells was on its way out. I ordered up
2 new high capacity batteries, spent a night in really cold
temperatures with the camera running constant for about 5 or 6 hours,
switched it on the next day and still managed a full day of picture
taking without needing to recharge the packs. Few days later i
recharged the batteries before going to bed, switched the camera on
next day and only got about 10 pictures before the camera died. I was
beginning to think it was me LOL. The only thing is, i dont have a
5D2, i have the 550d, and can now confirm that buy the looks of it
there may be a problem with the 550d draining power even whilst
switched off. I will try to keep an eye on it and see if i can narrow
down how / when / why it is caused.

thanks

Dave
> The main feature is full control over FPS, which was researched here:http://groups.google.com/group/ml-devel/browse_thread/thread/b1dcd030...
> files from Sztupy's zip (http://groups.google.com/group/ml-devel/browse_thread/thread/7bfc8b67...),
>  autoexec.bin
> 2253KViewDownload

I wasn't going to mention this, but it looks like I should.  The other day, I found my 60D's battery to be completely drained after I had only used it slightly after a full recharge the day previously.  So it sat unused, switched to "Off" for about 18 hours, but had no charge when I went to use it.  Has anyone else dealt with this?  Its the first time I've had such an issue happen in the eight years that I've shot, so I thought maybe it actually be something in the previous ML build.  I just wanted to check before I submitted it as a bug.

Gratz on the PHD Alex, well done mate!!
> The main feature is full control over FPS, which was researched here:http://groups.google.com/group/ml-devel/browse_thread/thread/b1dcd030...
> files from Sztupy's zip (http://groups.google.com/group/ml-devel/browse_thread/thread/7bfc8b67...),
>  autoexec.bin
> 2253KViewDownload

Well done on the PhD Alex! Definitely well deserved :-)

So it is stil not possible to get longer exposures than 1/30 even with
this very low fps?

Does anybody still have 1.0.8 for the 50D to share?

It is and has been possible since the first FPS override implementation . Set fps override to optimize for low FPS which will lock the shutter to 360degrees (1/FPS). So at .2 FPS you have a shutter speed of 5 sec.

Just a heads up. FPS control broken on the 500D. image breaks into a grid and the colors are all bad. :)

good to see that one can now shoot at 1080p24, too bad it's all messed up.

Thanks, Pel.

To unsubscribe from this group, send email to ml-devel+unsubscribe@googlegroups.com

Matthias, try this; let your camera record uninterrupted for at least 3 to 5 minutes, then try to stop the recording. My impression is that the movie file where the timelapse is going to be saved, must be more than a few frames of video, maybe it needs to be a second. If you are recording at 0.2fps, it will take a short while to reach enough frames. Or so I have found with my T2i.

thanks for the suggestion. i will also try this on my 550D the only problem i am facing as the frame rate is too low 0.2 so the screen is too much blurry that i cannot even focus anything. when i move my camera the screen update after a very long time and i cannot see anything. so too low fps is great feature but almost impossible to shoot footage.. if there is some other tricky way then please share..
Thanks..

Set your camera on manual focus and focus your shot before you change the mode to the slower FPS. At this point in your interest in photography/video you should have a good idea based in liveview that your view "should" be. You really must be at least somewhat familiar with what images your lenses produce as far as angle of view in your camera. As this mode is meant for timelapse video, most people would probably set the camera on a tripod, a dolly, or maybe a motor driven jib, but if you want to walk around and film things, just imagine what the liveview should look like and trust you are pointed in the correct direction. If you are trying this at 200mm zoom - you are not going to be able to accomplish much of anything beyond some abstract art; maybe! Beyond this, you should already know depth of field and how to shoot in manual mode with a small aperture to have everything in your scene, in focus. Good luck.

The FPS override settings should be saved in config file, only that
it's not enabled by default. So you only need to enable FPS override
at every startup, but not reconfigure it again.

How big is the drift speed? The adjustment from FPS override might be
too coarse for this.
> "unity2k" <quantumsp...@gmail.com> schrieb:
> --http://magiclantern.wikia.com/

Specially for your case i made TimerGen

http://g3gg0.de/projects/EOSTimerGen/

Start it, select your camera model and give the exact FPS you need.
select e.g. "5D Mark II" and enter your desired FPS.

for example if your camera number one is a little bit too slow and you have to increase it by 0.0009 FPS to 23.9769 then enter
this 23.9769 as your desired FPS and press enter/generate. (either , or . decimal separator, depending on your system locale)

it will output some possible results that are near your selected frame rate. choose the one you want to have.
on the right you will get a list of possible TimerA/TimerB combos to be entered in magic lantern.

e.g.

 Combos:
---------

A:1955 B:0512
A:1840 B:0544
A:0640 B:1564 (prefer)
A:0680 B:1472 (prefer)
A:1360 B:0736
A:1280 B:0782
A:1088 B:0920

the "(prefer)" marked ones you should try first. they don't obviously violate the "lowest seen" register value.
enter these in your camera and make some tests.

please report in if this works!
this is not tested at all :)

Thanks a lot UNITY2K i will try this what you have suggested.  before i have posted a 4fps video demo using my 550d at youtube and many people have seen that ( http://www.youtube.com/watch?v=3jhfAdjTya0 ) i hope soon i will produce some 0.2 fps video too :-)

cheers..

@Matthias von Mutius
oh, wait.
you have to subtract one from the printed values before entering in Magic Lantern.

i updated the EOSTimerGen to make it possible to measure your cameras reference clock.

just shoot a video of e.g. 10 or 20 minutes (filming some accurate reference clock) and count the frames.
enter these values in calibration dialog. voila - you get your cameras base clock to make exact frame rates.

if you want to sync two cameras, film at the same time the same reference clock.

http://g3gg0.de/projects/EOSTimerGen/

Time for some bugfixes :)

Most important are related to FPS control: it should now work properly
in all modes (including zoom/crop) and on all cameras. If you still
have problems, let me know.

For keeping the camera responsive at very low FPS, I've used a small
trick: first second of video is recorded at normal FPS. This lets you
stop recording right away, and avoids ERR70/80 on 550D/500D. You will
have to cut the first 25 or 30 frames in post.

Also refactored the waveform: it now uses 65K less memory and no
longer causes ERR70 on 500D.

A very important fix is on the audio side. Back then, there was a
digital filter / wind filter item in the audio menu, but the testers
did not find any difference in using them (for example,
http://vimeo.com/18724051 ). The bug was that ML was only turning the
filter off, but it was not re-enabling it (restart was needed).

Now, wind filter works and there's a clear difference in the audio
meters (since enabling the filter removes any DC offset too) and on
the frequency spectrum plot (see screenshots).

Also refactored the highlight recovery feature. Rather than overriding
Canon's digital ISO gain, it now alters this value. Reason: computing
Canon's digital gain is not 100% reliable (at some particular values,
like ISO 6400, Canon uses different tricks and taking care of all of
them is camera-specific and error-prone).

Recommended setting: full-stop ISO in Canon menu (100, 200 ... 1600)
and clipping point moved by 0.3 EV above Canon's default. This keeps
the benefits of 160x ISO (low shadow noise) while maintaining the
smooth highlight rolloff of full-stop ISOs. And, of course, ISO 80.

UniWB can also be assigned to half-shutter key (as a side effect, you
can choose to save the embedded JPEG either as UniWB or as normal WB).

Powersave should be OK now, especially on 5D (it had problems when
waking up with certain buttons, for example PLAY).

Lua scripting should work too. It was reported not to work in previous
version, but all I did was to recompile it. However, if I understood
well, the same plugin binary should be compatible with all ML version
as long as the API is not changed. Maybe I'm missing something.

For other (minor) fixes, see
https://bitbucket.org/hudson/magic-lantern/changesets .

Globaldraw / display preset suggestion is on the todo list, but bug
fixing had a higher priority.

Still having problems with incomplete shutdown on 5D. I've noticed the
problem was present since the first 5D unified builds, but I was not
able to find a pattern, because it happens very infrequently (roughly
once every 100 restarts, I think), and when trying to reproduce, the
problem seems to go away.

Some more timelapses with the extreme FPS feature:
http://www.youtube.com/watch?v=Ro1UcgvJgNs (0.2 fps)
www.youtube.com/watch?v=KIOxjYUgacM (0.25 fps)

It could be interesting to try one with extreme jello effect (of
course, with moving subjects, maybe cars or people). I have yet to try
this.

thanks a lot, alex :)

when i use wind filter  i can not record something everything is quiet

i copy the new autoexec to my sd card and switch on my 550d   first with internal mic then withe rode stereo mic i change the wind filter settings everything is ok.   but after pull out the external mix and reconnect there ist no sound recording  with internal mic  and external mic

only with wind filter off i can record something

Confirmed. The 60D also has this problem, but with slightly different
behavior (sound comes back after toggling wind filter).

With internal mic it seems to work on all cameras. The problem is only
with external mics on 550D and 60D. The spectrum plots were done on
5D2 with external mic.

By default, Canon firmware programs the AUDIO_IC_FIL1 like this when
recording:
on 5D:
- internal mic: 0x31 (enables wind filter, HPF and LPF)
- external mic: 0x1 (wind filter on)

on 60D (regardless of wind filter setting from Canon menu):
- internal mic: 0x31
- external mic: 0x11

on 550D:
- internal mic: 0x31
- external mic: 0

Also, on 60D, Canon is graying out wind filter control from Canon
menu. No idea why.

And to complete the mistery: on 60D I can't see any difference in
audio register configuration when wind filter is enabled or not (with
Canon audio controls). But in Audacity I can clearly see the
difference on the spectrum plot.

So, for now, I don't have any solution other than disabling wind
filter.

Not sure what I did, but on 60D seems fine now. I've disabled the wind
filter on 550D.

Meanwhile I've fixed a few more things (but I did not do any extensive testing):
- Highlight recovery: there was a typo which resulted in wrong
computations (it worked only in some particular cases).
- HDR timelapse should be fine (issue 1201). Hopefully it didn't break
HDR panoramas (lots of consecutive 9x1 brackets - didn't check for
that).
- 500D: ML graphics should no longer flicker on first clip (issues 1108/1045).

And, of course, CMOS temperature is now displayed in Celsius degrees
(see http://groups.google.com/group/ml-devel/browse_thread/thread/725ae6f424dd2917
).

Can anyone tell me what's all about with this lua plugin?!?!

> To unsubscribe from this group, send email to ml-devel+unsubscribe@googlegroups.com

Excellent work!!!
Thanks a lot Alex!!!

Carlos P
> (seehttp://groups.google.com/group/ml-devel/browse_thread/thread/725ae6f4...
>  Lua plugin.zip
> 97 KVerDescargar
>
>  autoexec.bin
> 2253 KVerDescargar

crop marks are not working
on the the 550D

--
--
ウェスリー D クラウデン
Wesley D Clouden
CIO/ Media Content Producer

Nappiee Corp Inc. // Mogulus LLC. // Twitter LLC

Qik.com , foursquare.com, InArtMedia.com
InYoFaceFilm.com, mogulus.com, Twitter.com

Skype //  Aol // Msn // Yahoo ID: Nappiee007

Wesley....@gmail.com
Dir: NYC 24/7: +1646 248-6680
***********************************

After trying for some time, works well & stable even with sound recording on(although I dont get any sound) on the 500D. Thanks to everyone who brought it this far

CROP MARKS ARE Not SHOWING Getting
IN cam ERROR MESSAGE

Oh thanks alex for the clarification about the sound. 

You have a 500D, so you can tell me how high it can go :)

On  setting it to 1920x1080 I get 24(from 20) & the exact fps shows that its not going beyond 20.196 fps.

Thanks.

We are still researching how to get higher FPS, but at this point I
can't promise anything.

On Tue, Apr 3, 2012 at 11:01 AM, Veeraragavan N

I can confirm that cropmarks ARE working on my 550D too

> ml-devel+unsubscribe@googlegroups.com

Actually I also encountered a problem which might be related to cropmarks on the 60D. I had 2 presets set up, and one of them had a cropmark. While I was in live view and switched the display preset to the one with the cropmark, I encountered an ERR70. After removing battery and trying again, it worked fine, even the cropmark. Haven't encountered it again so far.

Can you paste the crash log? It should be on the SD card (CRASH00.LOG).

> ml-devel+u...@googlegroups.com

Unfortunately the log file was deleted as I formatted the cards earlier on today as I had a shoot this evening. I'll do some more tests in the coming days and post any findings here (and this time if I encounter an error, I'll make sure to keep a copy of the log file!).

I just installed this .bin and am having issues with a group of characters being displayed in the bottom left third of the screen, Liveview or not. Still there even turning off Global Draw. It changes from time to time, 1-4 characters. Deleted config and still having issue.

What characters? Can you post a screenshot?

>> > ml-devel+u...@googlegroups.com

> ml-devel+u...@googlegroups.com

It seems that it displays 1000 in whole stop ISOs (100, 200, etc.) and c56 at ML-expanded ISOs. Earlier when set to AutoWB is was changing relatively rapidly with the changing exposure. At startup it displays 0, then changes soon after. This is with a 550d.

>> > ml-devel+unsubscribe@googlegroups.com

> ml-devel+unsubscribe@googlegroups.com

>> > ml-devel+unsubscribe@googlegroups.com

> ml-devel+unsubscribe@googlegroups.com

>> > ml-devel+unsubscribe@googlegroups.com

> ml-devel+unsubscribe@googlegroups.com

>> > ml-devel+unsubscribe@googlegroups.com

> ml-devel+unsubscribe@googlegroups.com

Are you sure you don't have the one from dropbox?

I've just downloaded it from this thread (2012Mar31.550D109, changeset
11c19cf78fbd+, built at 20:54:26) and it doesn't have the issue. It
was some debug info.

>> >> > ml-devel+u...@googlegroups.com

>> > ml-devel+u...@googlegroups.com

>> >> > ml-devel+u...@googlegroups.com

>> > ml-devel+u...@googlegroups.com

>> >> > ml-devel+u...@googlegroups.com

>> > ml-devel+u...@googlegroups.com

>> >> > ml-devel+u...@googlegroups.com

>> > ml-devel+u...@googlegroups.com

> ml-devel+u...@googlegroups.com

Yes, that's it, from bitbucket, I mixed up naming versions.

>> >> > ml-devel+unsubscribe@googlegroups.com

>> > ml-devel+unsubscribe@googlegroups.com

>> >> > ml-devel+unsubscribe@googlegroups.com

>> > ml-devel+unsubscribe@googlegroups.com

>> >> > ml-devel+unsubscribe@googlegroups.com

>> > ml-devel+unsubscribe@googlegroups.com

>> >> > ml-devel+unsubscribe@googlegroups.com

>> > ml-devel+unsubscribe@googlegroups.com

> ml-devel+unsubscribe@googlegroups.com

I took my 5D2 for test drive today. My videos are all shot at ISO 100 with clipping point -5 --> ISO 70. Most of videos that I shot today appears to have frame skipping. Has anyone experienced the same issue?

Can you send me a sample video? (raw footage, and log file if you have it)

VIDEO: [H264] 1920x1088 24bpp 23.976 fps 122502.3 kbps (14953.9 kbyte/s)

122502.3 kbps ???

It's too high bitrate:
http://vimeo.com/18024074
http://magiclantern.wikia.com/wiki/Bit_rate
and maybe change to CBR.

I've looked at the footage frame by frame, twice, and didn't notice
any skipped frames.

Consider getting a faster computer or a better video player.

Also, there is a reason why VBR is red in ML menu ;)