Electronic funhouse mirror - or - Calling all ray-tracers?

I want to build a funhouse mirror. I even have convoluted plans. But first, I'd like to try out a few electronic tests. Specifically, I want to find/build/acquire some program that will let you specify the surface of a funhouse mirror (at whatever resolution makes suff work, but basically, a set of x-y-z coordinates), then give it an image that is assumed to be directly in front of it, and see the resulting tranformation/map.

Does anyone have any brilliant ideas? Ray tracing seems relevant here, but I know next to nothing about it. Any clues on where to get started would be excellent.