Web Application Development Intro

Once upon a time..#

• Originally, Internet services were built based on two basic Internet protocols: FTP and TELNET
• FTP (File Transfer Protocol) was used for static information sources. It offered file-and-folder style access to remote data. Indexes, images, and navigation had to be stored in separate files and downloaded and viewed manually. Search engines at the time searched FTP sites.
• TELNET was used for interactive services. It offered a text-based console interface which users could use to interact with a server in dialog style.

Evolution of the web#

• The GOPHER protocol replaced FTP’s file-and-folder interface with interactive hierarchical menus and obtained some popularity
• The World Wide Web browser was invented by Tim Berners-Lee in 1989, based on applying the concepts of HyperText (a much older idea) to web navigation
• Berners-Lee’s browser, WorldWideWeb, was relatively unpopular at the time even compared to GOPHER.
• The Web’s popularity explosion came in 1993 with the release of Mosaic, the first strongly supported web browser with support for inline graphics and the Windows OS.

The key technologies of the web#

The “browser wars”#

• Web Browsers were commercial products with registration fees for much of the initial period of the web
• In addition, there was no standards body to rule what was or was not a valid web page
• Companies quickly realized that the browsers that the popular sites work on get the money
• This led to aggressive competition between browsers to add extra features, in the hope of having sites adopt them and thus requiring users to use those browsers
• This led to significant confusion and incompatibility, but also to very rapid development of extensions to the Web

Web Extension Technologies#

• Java. The first system for interactive content on web pages. Was revolutionary at the time, but considered slow and unwieldy once new options became available. Found its role as a language for web servers and desktop apps.
• JavaScript. Originally added to the Netscape browser to provide interactive functionality without the system load of Java. Actually had nothing to do with Java other than the idea of creating interactive content on web pages. JavaScript also had no standard, and multiple scripting languages competed until JavaScript was standardized as ECMAScript and incorporated into the newly emerging web standards documents.
• ActiveX. Microsoft’s initial competitor to Java, which would allow native programs to be downloaded from the web and run by web pages. Considered insecure, but did create the concept of the native Web plugin, which resulted in..
• Shockwave and Flash. Plugins for interactive graphics, which JavaScript did not support, created based on existing animation software. Flash especially boomed in popularity and became a major issue for browser authors and web standards organizations due to its ownership by Adobe. HTML5 added Canvas, its own standard for interactive graphics, which Adobe eventually moved to support.

How Web Browsers Work#

• You select a site to visit, either by entering a URL or clicking on a link in an existing page
• The content of a basic URL is broken down as follows: https://brookes.ac.uk/index.html

• A more complex URL can include additional parts: https://brookes.ac.uk/search.html

• When the file is retrieved, the web browser displays it according to the type of the file.

• Originally, this was determined by the file extension, but this proved to be a security risk. Modern web servers are expected to tell the browser the type of the file as part of the response protocol.

• Media files other than web pages (pictures, videos, etc) are displayed according to the browser’s capabilities.

• Other file types are either downloaded and saved or ignored, depending on security settings.

• HTML files (web pages) are displayed by the browser’s layout and rendering code.

• Usually, the HTML file will include inner references to other files which the browser must download to incorporate into  the web page. These include:

  • Media: images, videos, sound
  • JavaScript programs or parts of JavaScript programs
  • Further page layout and format information

• These files are obtained by separate requests which are made after the HTML is downloaded. Usually these would be to the same server, but they don’t have to be!

• The browser must run JavaScript and allow it to update the internal data structure the browser uses to represent the page (in JavaScript, called the Document Object), and change the drawn page to reflect the altered representation.

• The browser sends any data entered in forms back to the server when a link is followed, as part of the request for the file from that link.

• The “traditional” structure for a web server was:

• The path gives the file-and-folder location of the file to be downloaded.

  • This was for simplifying setting up servers, and for compatibility with FTP, which used only this method.

• The query gives the information which the user typed on a form and submitted on a previous page.

• Because simply sending a file cannot change in response to query data, responding to queries requires more sophisticated behavior, usually a program running on the server.

Dynamic Web Servers#

• Originally, a web server was any program that could respond to HTTP requests, which were relatively simple

• Originally for anything more complex than serving basic files, such as searching databases or responding to forms, you would write your own server

• As HTTP become more complex, load on the web increased, and security became more of a concern, it became unviable to write arbitrary web servers except in the most complex cases

• Instead, the standard web servers (such as Apache and Microsoft IIS) offered mechanisms for integrating other programs

• CGI (Common Gateway Interface): when a request for a dynamic resource is received, the server runs a program, passing it the request, and passes the program’s output to the user

  • Simplest method but not very scalable. Program has to completely start up and shut down once for every request

• FastCGI / Inner Server: program for responding to requests runs as a simpler server process. Web server converts incoming web request to a simpler request and passes it through to the inner server

  • Much more scalable. Inner server can be moved to a different computer to increase scaling even more. But developer still has to write a server, which can be a complex task and involve further scaling and synchronization concerns

• Module: web server includes an interpreter for a programming language. When a dynamic page is called, the server runs the interpreted program. Because the server controls the interpreter, it can manage startup and shutdown of the program and allow web specific code to be added to the program

  • Very efficient and convenient; entrenched the standard of using scripting languages for dynamic web sites
  • But can be insecure to have a large body of code in the web server, and can result in version conflicts between server modules and other languages; also, interpreted languages may  be too slow for very sophisticated websites

The Continuity Problem#

• When using an web site we generally expect to interact in a continuous session: we log in, stay logged in, add items to our cart which accumulate, and so on

• However, HTTP is not a session based protocol! Every request you send to a website is a completely separate request, and the server must match it up with previous requests you made, allowing for the fact that:

  • An arbitrary time may have passed since your last request
  • Any number of requests from others may have come in in the meantime
  • Your request may not come from the same IP address (dynamic ISP reallocation, mobiles joining / leaving Wi-Fi, etc)
  • Your request may not go to the same server (load balancing)

• Cookies were an attempt at solving the continuity problem.

• Created by JavaScript code and then automatically sent by the browser in every request made to a website.

• This makes it possible for the site server to match requests to previous requests by checking for the same cookie.

• However:

  • A cookie alone may not be secure as a hacker may steal the contents of a cookie via malware, or just try to guess it
  • Using cookies together with media retrieved from remote sites can allow a user’s requests to multiple sites to be matched with each other, which the EU deemed a violation of privacy

Is the Web a failure?#

• What the web was supposed to be:
  A distributed network of multiple information sources, worldwide, all communicating and connecting with each other without overall control.
• What the web actually became:
  A significantly centralized network providing connections to mostly a small number of dominant content providers, and a large number of smaller content providers integrated into even more dominant hosting providers
• Several web consortium initiatives (like the Semantic Web) ultimately failed because of this centralization