Well, although his power does seem to originate from the spot were there is a hole in his mask (that shows his eye), it could be explained that it's because it is the only spot, that his skin/body isn't covered with clothes or hair. Not to mention that his power affects only himself and the things he can touch (which doesn't seem so eye-like, if we compare it to Kakashi's jutsu that works on sight).
It's possible that it works that way, because the part around his right eye, is the only true part of his real body, that enables him to use the jutsu (so he can't use it through Zetsu's parts, as Zetsu can't use Kekkei Genkai, or any other jutsu (?) of the person it copied).
As it is the thread that concentrates on Tobi's space-time jutsu, allow me to explain on how it (possibly) works and some details about it.
Tobi has access to another dimension (perhaps it is his own creation), that he enters by creating a rift in space and from that space (the secret dimension), he can jump to any other place on earth. That is why his space-time jutsu isn't as fast as the 4th's Hokage, as he (the 4th) jumps directly from one place to another (or so it looks like), where he left his seal.
Side note: the dimension that Tobi enters, has no light, it's full of darkness, but there are some pillars that seems to be man made. It's possible that it is not another dimension, but a place that has no normal entrance, where light doesn't reach, like the inside of the moon.
His other space-time ability, is being ghost-like, capable of slipping through everything and any attack, up to about 5 minutes of duration. I believe it's thanks to leaving himself partially in another dimension (or place), that he can assume an ethereal-like form.
He can use only one jutsu at the time, that's why it's faster to partially leave himself in another dimension, rather than jumping all the way there and back. To even absorb himself or someone, he first must become solid and absorbing others takes longer.
Some of the source material:
