As recently as 1984, punched cards were still being used at a major institution (UWA Engineering Faculty). This offered a high degree of independence between the location of the user and the machine that the program was being run on (typically the user would be at the coffee shop or at home in bed), but had some major shortcomings.
Most sites have long been based around the character terminal. These are typically connected to the central host computer via serial lines running at between 1200 and 19200 baud. With fairly basic terminals, such as the DEC VT100, you have a single screen of 80 characters by 24 lines. on a serial line then a complete screen refresh would take between 1 and 16 seconds. Running a program on a remote machine is not difficult, all that you have to do is to log in to the remote machine (eg via telnet or rlogin) and run it. The output will come directly to your screen.
The situation would be somewhat different, however, if you were wanting to run graphics. A basic graphics terminal with 512 x 512 x 4 bit-planes resolution would require 128 kB of data for a complete screen refresh. Obviously this would make use of such a terminal rather tedious over a serial line, requiring over a minute for each screen refresh even at 19200 baud. Even if the terminal was directly attached to the host computer via ethernet (10 Megabit/sec), the network load would be unacceptable. With a more advanced graphics screen, eg 1280x1024x8 bit-planes requiring 1280 kB per screen, the situation would be even worse.
The solution to the above problem is to provide an intelligent terminal that understands how to draw graphics objects without requiring bit maps to be continuously sent across the network. In fact, X is so efficient that some vendors claim that you can run the X Window System across a 9600 baud serial line. (See the faq on X).