Research on designing graphical programming language

Reid Wang


As the bridge between human beings and computers, programming language is designed by computer scientists to replace the obscure machine language. Compared to binary code, programming language, thanks to its new features such as the application of natural language words and grammar, greatly enhance the efficiency of human – computer interaction.

However, for most computer users, the existent programming languages are still too abstract and difficult to understand, while the necessity for customizing functions has never disappeared. Further, the development of software links every modern industry with computer science, thus making higher requirement to citizens for better understanding of programming language.

This paper intends to discuss whether graphical programming language, as a possible form of future programming language, can help ordinary users study and use programming language in a more friendly and riveting way by analyzing the key principles of designing programming language that is valuable for daily use.


From vacuum tubes and transistors age to integrated circuits age, the computing power of computers has ballooned during the short 70 years. Computer itself also evolves from the gigantic machine occupying a whole room to the laptop and tablet that can be carried by people anywhere. However, no matter what form in which computers appear, the basic principle of electronic computer design has never changed. Compared to the first electronic computer born in Pennsylvania, though modern computers have already been manufactured with new technologies including graphical operating system, highly interactive software and powerful self-learning ability that can not be imagined by people several decades ago, they still can not work without the combination of 0 and 1 representing the basic logic unit like “and” or “not”. So the problem is that computer is essentially a machine with unique mechanism based on binary code which is totally different from human beings’ natural language. Although the early computer scientists can make instructions to computers using binary code directly, it is impractical for every developer to learn such kind of thing not only because of the high leaning cost but also the complexity of modern software. Therefore, a new form of language that can both embody the logic of computer and contain the features of natural language is needed. That is programming language.

This paper will discuss the relationship between programming language and graphical design. On the whole, this paper will be divided into four main parts. Part one will focus on computer and programming language, including the computation mechanism of computer system and key features of programming language. Part two will expand the topic to the relationship between computers and human beings as well as the information flow from machines to human beings. Part three includes the application of design principles in designing graphical programming language with a few cases. Part four will talk about the social and business impact of popularizing graphical programming language.

1.Programming language and computers

1) Some basic concepts
Before meticulously researching the new form of programming language, it is imperative to talk a little about the execution process and working mechanism of programming language. Most of computer users have heard that computers are based on binary code, but few of them really understand what “binary” really means.

One critical concept about binary system is Boolean algebra. This theory introduced by George Boole describes logic system in a mathematical way and lays foundation for the binary system of modern computers. The core part of Boole’s theory is the variables with two possible values representing true and false respectively. Therefore the system consisting of n variables can generate 2n possible states. A similar system is Morse code which uses dot and dash to represent English characters and numbers. Back to modern computers, by controlling the value of each logic unit, computers can execute complex instructions since the billions of transistors integrated in a CPU can combine with each other to produce innumerable possibilities.


A binary-tree model to illustrate the mechanism of Morse code

Programming language itself just translates the binary code to people and people’s words to computers. Just as we have to combine words and words together to explicitly express what we mean, we need a collection of certain programming language words for executing our instructions and that is program. The translation process is also executed by a program called compiler. According to the classical book about compiler, Compilers, Principles, Techniques and Tools, compiler is defined as “a program that reads a program written in one language – the source language – and translates it into an equivalent program in another language – the target language”.

Though all programming languages serve like translators between human beings and computers, considering the gap between binary code and human beings cognition, the programming languages with more features of human beings’ natural languages are easily accepted by learners in practice. However, the efficiency of a programming language is usually inversely proportional to its legibility.

2) Key features of programming language
Today one of the most famous disputable topic among programmers is which programming language, from the most complex ones like C++ to the relatively simple ones like javascript, is the best one in the world. It is hard to rank the mature commercial programming languages because every language has its own pros and cons. But there are some common features of prevalent programming languages and those features are determined by computer itself.

  • Natural-language-based

In light of the uniqueness of computer’s logic, the mature programming languages are almost based on human beings’ natural languages, especially English. It is not difficult to understand why to say so. For users, there is no need to describe the working process of computers. For example, users are not interested in what happens in BIOS or hardware when they start up a computer and it is enough to show them where power button is. To eliminate ambiguity, a battery-like icon is often presented besides a power button. For developers, programming language allows them to input single words to realize complex functions. Like the power button, programmers only need to call certain key words instead of exactly knowing everything happening within CPU, memory and hardware. In a simple program like “Hello World”, every word has its own meaning and the meanings can be easily accepted by learners because they are tightly related to their original meanings in English. For instance, the key word “printf” just means printing certain contents on screen.


Classical “Hello World” program in C

The relationship between programming language and natural language can be explained reversely, too. Natural language helps programming language build its meaning system and programming language in turn helps computers understand what words of natural language mean. As mentioned above, Morse code adopts a special way to simply signify characters and numbers. The way to make computer understand words and numbers is similar. An encoding system, like ASCII, will construct mapping relationship between words and special code that can be read by computers, such as binary code or hexadecimal code.

  • Modular

If we really begin to learn a main stream programming language, we can find that most of them, no matter what perspective we use to view them, is black-boxed and modular-designed.

First of all, most programming languages are based on functions containing relatively independent code called code blocks. Branch statement, of which form is if A, then B, else C, is a great example. In program, A, B and C are clear because each part will be strictly enclosed in brace. Maybe programmers still have to write code line by line, but it is much more convenient for eliminating bugs because by modularizing each logic part, programmers can check the programs block by block instead of word by word.

What’s more, a program can not run without the support of operating system. That is why software developed for Windows can not run on Mac os. To be specific, the code inputted on visible interface itself does not mean anything to computers because computers are not manufactured with recognizing key words of popular programming languages. Actually, what stipulates that “printf” means printing words on screen or “int” indicates integer is exterior files called head files. In the above example, what makes key words like “printf” work is the head file “stdio.h” listed at first. For programmers, it is unnecessary for them to know what is contained in the head file (in fact stdio.h is about standard I/O function). Common head files together constitute standard libraries. Obviously, standard libraries only tell programmers what is available to them, but not why it is available. That is why program is black-boxed.

  • Hierarchical

No matter what language we use, ultimately programming is about communicating with computers. People themselves do not have the ability of controlling electronic states of transistors directly. Therefore, computer is a huge black-box system for human beings.


Natural language to machine langugae – interface tp hardware

Machine language is the language of hardware and natural language is the language of human beings, each representing a different logic. A programming language with more features of natural language can be understood by people more easily but computers will spend more time converting it into machine code because calling standard libraries and interpreting syntax can be time-consuming. Low-level programming languages that are close to hardware, like assembly language, are often highly efficient but can hardly understood by people. From this perspective, it is reasonable to say that programming language is hierarchical.

2. Human – computer interaction

1) Information flow
Shannon’s traditional information theory has already built a model for analyzing what happens when people communicate with computers. For modern computer users, they are accustomed to receiving information from the pixel-based screen. However, the problem is that information theory can not explain how brain processes information since the destination of information – human brain is essentially a black-box system in information theory. Another serious flaw is that information theory ignores the effect of information form.


Shannon’s information theory model

In fact, two things interesting of Shannon’s model about programming is exactly the information source part and destination part. The former one is about computers and interface and the latter one is about human beings.

For human beings, the form of information can profoundly influence the final effect of communication. A famous theory concerning this is from Neil Postsman, who regards media as metaphor. To be specific, Postsman criticizes that TV, as a new form of media, distracts people from paying attention to content compared to newspaper. The statement, however, to some extent indicates that images are more attractive to human beings.

2) Human – computer interface
Now that programming language is designed for people and used by people, the design principles used for daily objects are also applicable when evaluating a programming language. Just as natural languages which update every year with thousands of new words and grammars, programming languages also head towards the direction of supporting more key words, functions and libraries. However, what happens with the development of programming languages is that learning and using programming languages is not easier than decades ago. On the contrary, the operating systems, from DOS to Windows, greatly lower the cost of using computers for people.

Compared to DOS, Windows 95, the first generation of Windows operating system that can work independently, introduced a brand-new way of communicating with computers. Users don’t need to input word-based instructions and wait for feedback shown on monochrome interface, instead, in order to implement certain function, they only need to click buttons or icons with clear words or images. The following version of Windows operating systems optimize user experience, but never overturn the form of graphical interface. For programming language, the history of operating system at least indicates one thing that graphical design may help users better understand what they can and should do when they contact a new product. Therefore, graphical programming language has the potential to revolutionize the way people study computer science and develop software.




From words to graphics – the history of Windows interface

3. Designing graphical programming language

1) Interface design – affordance, constraint and feedback
To design a graphical programming language, the first step is to design an interface that can clearly tell users what to do and what they can achieve. For programming learners, two difficult things is remembering what key words mean and designing a proper algorithm for certain goals. So for graphical programming language designers, they have to meet the basic requirements from users.

Blockly, the online graphical programming project developed by Google, makes a good example to solve the problem. The whole interface of Google Blockly is divided into three parts. The left part provide users with selections of programming modules available for building their own projects. To help users understand the effect of available modules and find out the modules they need quickly, designers of Blockly categorize all modules according to their function. The middle part is the main area that allows users to constitute their own programs. By dragging the modules and modifying values of the parameters in the light-colored areas, users can freely decide what their programs are like. The right part just shows the equivalent source code. For users who are interested in learning basic concepts in a more graphical way, the source code can be helpful.

Just like Lego, Google Blockly offers similar affordance, constraint and feedback to users. By observing the margin shape of each module and reading the instructions, users can establish preliminary impression of programming. When a module is dragged to the right location that can make the whole program work, it will shine and create special sound effect. To prevent users from making mistakes, module will not make response to users when it is placed at wrong place.


Interface of Google Blockly

Considering one of the most important characteristics of programming language is modularity, the interface design of Google Blockly is perfect for people to understand what modularity means. In fact, a good design for graphical programming language interface not only should be user-friendly, but also should reflect the features of object programming language.

One feasible strategy for designing a graphical programming language is gamfication. Of course, gamfication does not mean designing a game but absorbing the advantages of game as a new form media that create a sense of immersion, which can be explained as “not interrupting us all the time to tell us it does not matter”.

So the first question is what is game what makes game different from other forms of media. Generally speaking, game cab be defined as “a problem-solving activity, approached by a playful attitude” by Jesse Schell. If we narrow the range down to screen-based games, we can find what makes game intriguing and playful is its interaction with people.

Here another question emerges – what does interaction mean to players? To answer this question, Richard Bartle comes up with a model that divides players into four main types – killers, achievers, socialists and explorers. For players, Although Bartle’s system is initially used to explain the players’ psychology in MUD(Multi-user dungeon) games, considering the interaction style of games does not intrinsically change, Bartle’s theory is still valuable for researching modern games.


Bartle’s theory of players

Back to gamfication of programming language, the first thing we have to do is to figure out what role programmers or learners play in Bartle’s system. No matter full-fledged programmers or inexperienced beginners, what they face is unknown and waited to be created. Therefore, we can easily categorize them as explorer. What’s more, gamfication means response from computers is important as well. Considering this, taking users of graphical programming language as achievers is necessary.

So now our goal is to design a fully interactive and immediately respondent programming language with attractive interface, of course. Here is a great case of a programming language that meets all these requirements – Scratch.


Interface of Scratch

Scratch is an interesting teaching programming language developed by MIT. Just like Google Blockly, it adopts the Lego-like design to reflect the inner logical structure like loop or branch in the program. What is different is that it does not strictly simulate the traditional programming languages but uses a pseudo-code and natural-language style to create interactive effect. In the example shown in the above picture, one amazing thing is that the property and action of the cartoon character, whose parameters can be adjusted by learners, is labeled with different colors. By doing this, learners can quickly understand the definition and meaning of statement or function. The execution result of the Lego-like code is that the animate characters can move, stop or change costumes precisely according to what learners have inputted.

As mentioned above, users of graphical programming languages are best regarded as explorer and achiever. Apparently, the users of Scratch can easily be motivated because their work can be directly reflected as an animation that can only be achieved by proficient programmers without Scratch. Also, Scratch provides users a complete community service allowing users to upload their own projects, thus encouraging learners to explore more possibilities using Scratch language.


Scratch’s powerful community

3) Efficiency
There have been many graphical programming languages that focus on learners and light users, like product managers and UI designers, who have the demand for writing short scripts occasionally, but professional programmers, hackers and computer science students still prefer to program in old way. Be it Blockly or Scratch, those graphical programming languages are designed as teaching tool instead of production tool. Compared to traditional programming languages, the existing graphical programming languages can not solve some problems like:

  • Low efficiency of complex program

Low efficiency of graphical programming language can be explained from two perspectives. On the one hand, graphical interface means calling more exterior functions, in other word, involving more black-box systems. It will definitely lower the efficiency of executing program. On the other hand, the industrial programs can be hundreds times as complicated as the programs developed by graphical programming languages. It will be inconvenient for programmers to read because too many geometric figures can distract programmers from what the program is really about.

  • Programming is not all about programming languages

A prevalent misunderstanding people have of programming is that learning programming language equals with learning programming. It is ridiculous to say that anyone who knows English is novelist and that is the same to programmers. Graphical programming language may be able to help learners understand the basic concept of Boole algebra, but it can not teach them how to use it. Therefore, graphical programming language itself is not enough to help people understand programming and computer science.

Graphical programming language is an excellent teaching tool but not a great production tool now. To develop graphical programming languages that can replace traditional programming languages, designers have to jump out of the circle of existing programming languages. However, the co-existence of graphical programming languages and traditional programming languages may be the best situation considering the development of computers.

4. Social – technical system

1) Computational thinking
Although graphical programming language itself is not enough for anyone who aspires to enter IT industry, the popularization of it can propel people to form attitude of computational thinking, which is interpreted by Jeanette Wing as “computational thinking involves solving problems, designing systems, and understanding human behavior, by drawing on the concepts fundamental to computer science”.

So how to explain “ the concepts fundamental to computer science”? Back to the learning process with Scratch language. During the learning process, learners will observe, experiment, analyze, summarize and introspect. Also, to realize the object like “making the character head upwards for three steps in yellow costume” , they will learn to break down the conundrum into some sub-problems like “head upwards”, “three steps” and “yellow costume”. In general, computational thinking is an intelligent, rational and efficient way of thinking.

Graphical programming language unveils the mysterious programming language to ordinary people. We do not need everyone to be eminent programmer who can finish a complete project independently, but computational thinking should be accepted by every modern civilian.

2) Non-elitism
Another interesting thing relative to the popularization of graphical programming language is that user-friendly interface and appealing interactivity can effectively lower the barrier for developers, or more precisely, small-scale developing teams or individual developers. Maybe those developers are incapable of developing any large software targeting millions of users, but the opportunity for them is that need can be personalized. Sometimes the need can be so specific or minor that big companies often ignore the voice from those users. If we browse the plug-in center of Chrome, we can find many functions beyond our imagination.

Under such a circumstance, developers can be users themselves. Users with peculiar needs will develop small program or write a few lines of scripts themselves and then share with others online. For those developers, they don’t need to learn computer science courses systematically. So the best programming language for them is the simplest and most visualized one.

I would like to call the emergence of individual developers or non-proficient developers non-elitism in Internet age. Non-elitism means that anyone who has special interest can develop his or her own application, build personal website or create art work, thus forming community with single topic and strong user engagement.


As a form of language that plays important role in human-computer interaction, programming language guides people how to communicate with computers and de-blackbox the complex digital system in our life. However, the abstraction and complexity of main stream programming languages prevent ordinary people from learning them. Graphical programming language, compared to traditional programming language, is accepted by ordinary people more easily because of its clearness and interactivity in education field. Nevertheless, it is still hard to make verdict on the future of graphical programming language in working field.


Adams, Ernest. “The designer’s notebook: Postmodernism and the 3 types of immersion.” Retrieved 1.5 (2004): 2015.

Aho, Alfred V., Ravi Sethi, and Jeffrey D. Ullman. Compilers, Principles, Techniques. Addison wesley, 1986.

Anderson, Chris. “The long tail.” Wired magazine 12.10 (2004): 170-177.

Arthur, W. B. (2011). The Nature of Technology: What It Is and How It Evolves (Reprint edition). New York: Free Press.

Bartle, Richard. “Hearts, clubs, diamonds, spades: Players who suit MUDs.” Journal of MUD research 1.1 (1996): 19.

Bodker, Susanne. “When second wave HCI meets third wave challenges.”Proceedings of the 4th Nordic conference on Human-computer interaction: changing roles. ACM, 2006.

Boole, George. An investigation of the laws of thought: on which are founded the mathematical theories of logic and probabilities. Dover Publications, 1854.

Bryant, Randal, and O’Hallaron David Richard.Computer systems: a programmer’s perspective. Vol. 281. Upper Saddle River: Prentice Hall, 2003.

Fangohr, Hans. “A comparison of C, MATLAB, and Python as teaching languages in engineering.” International Conference on Computational Science. Springer Berlin Heidelberg, 2004.

Jeannette Wing, “Computational Thinking.” Communications of the ACM 49, no. 3 (March 2006): 33–35.

Manovich, Lev. Software takes command. Vol. 5. A&C Black, 2013.

Norman, Donald. The Design of Everyday Things: Revised and Expanded Edition. Rev Expedition.
New York: Basic Books, 2013.

Postman, Neil. Amusing ourselves to death: Public discourse in the age of show business. Penguin, 2006.

Prechelt, Lutz. “An empirical comparison of seven programming languages.”Computer 33.10 (2000): 23-29.

Rochkind, Marc J. Advanced UNIX programming. Pearson Education, 2004.

Schell, Jesse. The Art of Game Design: A Book of Lenses. 2nd edition. Natick: A K Peters/CRC Press, 2014. Print

Shannon, Claude Elwood. “A mathematical theory of communication.” ACM SIGMOBILE Mobile Computing and Communications Review 5.1 (2001): 3-55