We propose a new approach to competitive analysis in online scheduling by introducing the concept of online approximation schemes. Such a scheme algorithmically constructs an online algorithm with a competitive ratio arbitrarily close to the best possible competitive ratio for any online algorithm. Also, it provides algorithmic means to compute the actual value of the best possible competitive ratio up to an arbitrary accuracy. We study the problem of scheduling jobs online to minimize the weighted sum of completion times on parallel, related, and unrelated machines. By constructing online approximation schemes we derive both deterministic and randomized algorithms which are almost best possible among all online algorithms of the respective settings. We also generalize our techniques to arbitrary monomial cost functions and apply them to the makespan objective. Our method relies on an abstract characterization of online algorithms combined with various simplifications and transformations.