certain families chose, it seems, to specialize in chakra-manipulation and created their own ninja clans with their very own chakra technique. which in turn probably gave rise to ninjutsu, genjutsu, and taijutsu.
remember however, that what the sage taught wasn;t ninjutsu but something called ninshu, as a way of peace.. my take on it is that he was trying to enlighten people to what they really are, energy, chakra, and therefore we're all one on certain levels..
but the sage himself only had two sons, which lead to a few clans like the uchiha, senju, uzumaki, and hyuuga(i'm sure)..
the cloud brothers seemed to have been DISTANTLY related to the sage as well, probably the younger son. everyone else(considering how many different skin-tones, hair color, and eye colors exist), no blood-tie to the sage or his sons..