There does not seem to be a universal set of signals agreed on by all, and some signals/meanings only make sense for certain kinds of paddling and not for others, but here is a basic set that some kayakers use and can serve you well. Consider also that the signals employing a paddle may be more useful given the kinds of distance separation that can occur in downwinders, though that will not be an option if you have lost your paddle.
From tsunami rangers article (with a few additional comments)