Don't want to give iMovie for iPad the ability to know your location? Well then iMovie for iPad won't let you see your Camera Roll. No, it's not being petulant it's probably just an odd result of the way iOS handles permissions.
Camera Roll may contain geo-location data in the pictures and videos you've taken. If iMovie accesses Camera Roll, it gets access to that location data. My guess is that Apple feels it then needs permission to "see" the location data.
Personally I regard location data as highly private and see no reason why Camera or iMovie should have it unless I specifically turn it on while traveling to geotag something for future reference. So it's a little annoying that I can't access my non-geo tagged pictures and video in my location-denied iMovie, but hopefully Apple fixes the granularity on this in an update so tin-foil hat wearing privacy throwbacks by myself can scratch one more conspiracy off our walls... er... list.