Friday, October 28, 2005
Update on iPod findings
Well, I've been messing with video for the iPod. I was taking notes, and then turned them into a pseudo-blog-post. I did this because of the ffmpeg frontend that I'm writing, as previously mentioned. I need to figure out how to encode my own video that will play on the thing, and I need to figure out what the out-of-spec limits of the iPod are, and how it handles various situations. That sort of thing. Here they are:
iPod Video findings:
- The iPod defaults to widescreen mode where it letterboxes widescreen content. It does this for any widescreen aspect ratio, not just 16:9
- You can feed the iPod any size video with at most 230,400 pixels. The actual specs Apple gives of max 480x480 is dumbed down. BTW, 480x480=230,400
- The max 4:3 res that can be handled that has whole number sizes is 552x414. That's actually a bit less than 230,400 so it gives a nice little buffer in case of 1 or 2 pixel sizing errors.
- The iPod cannot handle anamorphic content (It crops), so if the player detects less wide than 4:3 (like 1:1) then just scale it to 4:3. If size is too big just scale down to 552x414
- 4:3 content: If greater than 552x414, scale down. If not, leave res as-is.
- Widescreen content: If less pixels than 230,400 leave as-is. If not, figure out how much it needs to be scaled down to stay under 230,400
Much video will NOT need to be scaled down. For example, my DVD rips of Babylon 5 are encoded at 576x320. This is only 184,200 pixels, so I can just transcode the format without touching the resolution, and the iPod will letterbox it.
Why would I want to encode at higher resolutions than 320xSomething since that is the iPod native screen size? Well, because the higher resolutions are going to look sharper than just scaling it. You get a bit of moire patterns though, but it's worth it. Also, the iPod Video can output to TV via S-Video or Composite with the right cables, and it might only output at 320x240 but it might output the actual res too. I didn't buy the cables so I can't experiment.
While the ipod supports up to 2.5mbit for MPEG-4, if you actually set the res to the admittedly outside spec res of 552x414 and then run at 2.5 megabits, you're probably going to be stressing the video decoder some. Since the screen is still only 320x240, there is NO point encoding video at that kind of bitrate. I'm doing my test videos all at 1mbit, and I can't really see any visible artifacts. In the darker bits I can sort of make out macroblocks, but in normal brightness I can see none. So maybe you would try 1.5mbit, but not really any higher. I've set 1.5mbit as the max bitrate in my frontend's iPod profile, and I'm trying to decide if the default should be .75 megabit or 1 megabit.
The problem with encoding video by hand with ffmpeg is that you're going to have to figure all this stuff out. I could write a human script to run through for people to figure out what resolution to use, but it's going to be annoying to run through and it'll involve some annoyingly long math for some situations. The frontend should be able to handle all these situations by itself by default. Oddly enough the few other iPod converters that I found (One of which was based on ffmpeg too, the other used it's own encoder) did not do any of this on their own. They just scale to 320x240, and don't even correct the aspect ratio. They just assume the content is 4:3 and directly scale from anything to 320x240.
So, encoding using my frontend should be as simple as to open file(s), select iPod as profile, select bitrate, select resolution (320x240 or automatic max) and hit encode. Audio settings are handled by the program, it selects an audio bitrate based on your video bitrate. Yeah, I could let the user select that too, but really there is no need, since that adds useless complexity. They can make their own profile if they want.
I haven't checked how different framerates are handled yet. My Babylon 5 stuff is 24FPS (Well, 23.98). Hopefully 30FPS (Well, 29.97) is handled properly too. I'll experiment further later.
EDIT: If you're wondering why I'm not worrying about h.264, that's because there is not really any benefits to using it on the iPod. It is more limited than MPEG-4 so has lower quality. It encodes several times slower. It's really only useful for lower-bitrate lower-quality videos, and iPods are huge (30GB minimum now). YAGNI.
iPod Video findings:
- The iPod defaults to widescreen mode where it letterboxes widescreen content. It does this for any widescreen aspect ratio, not just 16:9
- You can feed the iPod any size video with at most 230,400 pixels. The actual specs Apple gives of max 480x480 is dumbed down. BTW, 480x480=230,400
- The max 4:3 res that can be handled that has whole number sizes is 552x414. That's actually a bit less than 230,400 so it gives a nice little buffer in case of 1 or 2 pixel sizing errors.
- The iPod cannot handle anamorphic content (It crops), so if the player detects less wide than 4:3 (like 1:1) then just scale it to 4:3. If size is too big just scale down to 552x414
- 4:3 content: If greater than 552x414, scale down. If not, leave res as-is.
- Widescreen content: If less pixels than 230,400 leave as-is. If not, figure out how much it needs to be scaled down to stay under 230,400
Much video will NOT need to be scaled down. For example, my DVD rips of Babylon 5 are encoded at 576x320. This is only 184,200 pixels, so I can just transcode the format without touching the resolution, and the iPod will letterbox it.
Why would I want to encode at higher resolutions than 320xSomething since that is the iPod native screen size? Well, because the higher resolutions are going to look sharper than just scaling it. You get a bit of moire patterns though, but it's worth it. Also, the iPod Video can output to TV via S-Video or Composite with the right cables, and it might only output at 320x240 but it might output the actual res too. I didn't buy the cables so I can't experiment.
While the ipod supports up to 2.5mbit for MPEG-4, if you actually set the res to the admittedly outside spec res of 552x414 and then run at 2.5 megabits, you're probably going to be stressing the video decoder some. Since the screen is still only 320x240, there is NO point encoding video at that kind of bitrate. I'm doing my test videos all at 1mbit, and I can't really see any visible artifacts. In the darker bits I can sort of make out macroblocks, but in normal brightness I can see none. So maybe you would try 1.5mbit, but not really any higher. I've set 1.5mbit as the max bitrate in my frontend's iPod profile, and I'm trying to decide if the default should be .75 megabit or 1 megabit.
The problem with encoding video by hand with ffmpeg is that you're going to have to figure all this stuff out. I could write a human script to run through for people to figure out what resolution to use, but it's going to be annoying to run through and it'll involve some annoyingly long math for some situations. The frontend should be able to handle all these situations by itself by default. Oddly enough the few other iPod converters that I found (One of which was based on ffmpeg too, the other used it's own encoder) did not do any of this on their own. They just scale to 320x240, and don't even correct the aspect ratio. They just assume the content is 4:3 and directly scale from anything to 320x240.
So, encoding using my frontend should be as simple as to open file(s), select iPod as profile, select bitrate, select resolution (320x240 or automatic max) and hit encode. Audio settings are handled by the program, it selects an audio bitrate based on your video bitrate. Yeah, I could let the user select that too, but really there is no need, since that adds useless complexity. They can make their own profile if they want.
I haven't checked how different framerates are handled yet. My Babylon 5 stuff is 24FPS (Well, 23.98). Hopefully 30FPS (Well, 29.97) is handled properly too. I'll experiment further later.
EDIT: If you're wondering why I'm not worrying about h.264, that's because there is not really any benefits to using it on the iPod. It is more limited than MPEG-4 so has lower quality. It encodes several times slower. It's really only useful for lower-bitrate lower-quality videos, and iPods are huge (30GB minimum now). YAGNI.
