ESP01-8266
The ESP-01S-8266 is based on the WiFi chip 8266 developed by Espressif. A 32 bit micro controller (MCU) is integrated in the chip. The external dimensions of the module are 14.3mm * 24.8mm * 3mm, which is very small. Many types of boards have been developed based on this chip. The ESP-01S is an example of this.
Basic technical data of the ESP-01S:
– Supply Voltage 3.3 v typical
– 512KB Flash Memory
– 1 MB program memory
Objective of the experiment
Because the ESP01S has a built-in MCU with the necessary memory and a WiFi module, it must be possible to build a web server with it. That this is possible is proven by the many successful experiments that can be found on the internet.
In general, these are descriptions with an LED web-based manipulate or presenting temperature and humidity in a web page with the sensor DHT11.
In all examples, the module was set up as a server, which can be queried by the client (browser) for (desired) data and as a result the data is displayed in the client (browser).
This is a little bit more complicated if you want to display the current GMT or Local Time in the client (browser), which is based on the Network Time Protocol (NTP). After all, the board need to be both a client and a web server at same time. Nevertheless, the technical description of the ESP01S shows it should be possible.
A second reason (perhaps just as important) was to get more knowledge of HTML, XHTML C++ (Arduino derivative) JAVA servlet and AJAX.
Furthermore, the module had to be able to be programmed via the WiFi network. The so-called OTA programming (Over The Air programming).
How does the World Wide Web work
Asynchronous client-server process
Of course, I assume that people are familiar with the process behind the World Wide Web. But for a good understanding of working asynchronously, let’s take a little look at the basics of the process:
– Type the address of the desired website in your browser in plain text
– The browser looks up for the IP address (DNS request) of the requested website
– The browser uses the obtained IP address to search for the desired page of the requested web server on the World Wide Web
– The web server returns the requested page to the web browser
– The web browser displays the page on your screen
A few notes on this.
When the server receives a request from the client (browser), it sends what the client ask for in a data structure that the client has requested. The client then presented the requested information based on internal rules. This means that the data obtained, can also be presented differently in the different browsers. In short, the server does not model the data, the browser does. So the “intelligence” is in the browser.
An important observation is therefore that the server does nothing if it is not requested!!
Based on the protocol described above, this means that if you want to control an LED or display the temperature or humidity, you must always query the server for updated data. After all, the server from which the data is to come does nothing without being asked.
This is not only cumbersome but also inconvenient. This is where AJAX comes into play.
Asynchronous client-server process
AJAX client-server process
AJAX is used for the asynchronous processing of the client-server process.
AJAX stands for “Asynchronous JavaScript and XML”. It can be used to update part of the web page without reloading the associated page. The client does this by “spontaneously” (or not) requesting, receiving and processing data from the server. The function of AJAX is to update web content asynchronously. This means that a user’s browser does not need to refresh the entire web page when only part of the content on the page needs to be updated.
AJAX is een combinatie van:
– XML (Extensible Markup Language)
– Javascript en HTML
Where HTML is used to build the web page in the browser, XML is used to transport data.
When a client visits a web page and an event occurs, the JavaScript creates an XMLHttpRequest-object, which then transfers information in XML format between a browser and a web server. The XMLHttpRequest-object sends a request to the web server to obtain updated data. The web server processes the request, generates a response, and sends it back to the browser. The browser then uses JavaScript to process the response and display it on the web page without reload the entire page.
JavaScript.
JavaScript does the update process in AJAX. It requests for updated content from a web page. The web page is formatted in XML to make it understandable. JavaScript then refreshes the content of the web page. The user get the updated page and can view it.
The AJAX process in order of events:
1. An event occurs on a web page in the browser (the page is loaded, e.g. a button is clicked)
2. An XMLHttpRequest-object is created by the JavaScript
3. The XMLHttpRequest-object in the browser sends a request to a web server
4. The web server processes that request
5. The web server sends a response back to the browser
6. The response is read in the browser by the JavaScript
7. The appropriate action (such as object update) is performed by the JavaScript
What is the NTP server
AJAX NTP request
The Network Time Protocol (NTP) is a network protocol for clock synchronization between computer systems over variable latency packet-switched data networks. This is quite a mouthful. What it meant here is that when you request fot the time via the internet, time elapses before you receive the result. That is, the result no longer corresponds to the real time. The NTP protocol ensures that this time delay is taken into account.
The protocol runs on an NTP server formed by pool.ntp.org.
As pool.ntp.org will allocate you time servers from all over the world the quality of the time will not be ideal as noted. You get a somewhat better result when you use the continental zones (for example europe.pool.ntp.org), and an even better quality when you use the land zone such as nl.pool.ntp.org for the Netherlands. But more on that later. For more information about the NTP protocol click here.
The NTP time in the web browser
AJAX NTP request
If you take a closer look at the sequence of events on pressing a button in the “browser page” you will notice that the action/event is based on an action in the browser. So the client. The client does something and the server responds.
This is not the case when requesting the NTP time. Here the client independently takes the lead (often time-controlled) and ensures an asynchronous action.
Using the ESP01 module
The ESP8266 can be controlled from the local Wi-Fi network or from the Internet (after port forwarding). The ESP01 module has GPIO pins that can be programmed to turn ON or OFF an LED or a relay over the internet. The module can be programmed using an Arduino/USB to TTL converter via the serial pins (RX, TX).
Technical data of the module EPS01 module
Pins are arranged in two rows of 4 pins. Some models have a pin description on the PCB. This makes it easy to recognize the correct pin.
On the top row you will find the following pins from left to right:
1. GND (power supply ground)
2. GPIO 2 (digital I/O programmable)
3. GPIO 0 (digital I/O programmable, also used for BOOT modes)
4. RX – UART receive channel
Modifications of UART module
UART
To put ESP-01 in flash mode via the UART, you need to connect GPIO 0 to GND.
The UART module does not do that by itself. To make this possible, you can glue a push button to the UART module and connect this button to the GPIO 0 and GND pin of the UART module. See picture.
The procedure is now as follows.
Program mode
Open the Arduino IDE and insert the UART module with the push button pressed into the computer’s USB port.
In the Arduino IDE the USB port is now visible to use. In other words, the ESP01 is in “program mode”.
Run mode
If you now want to put the ESP01 in “Run mode”, remove the UART module from the USB port and put it back “WITHOUT” pressing the push button. The ESP-01 is now in “Run mode”.
USB port accessible under LINUX (Ubuntu)
Another problem you may encounter is the UART chip on the UART module.
There are two different UART modules in circulation. One with the chip CH340 or CH341 and one with the chip CP1202.
In the LINUX kernel, with the April 2022 firmware update, the firmware for the China clone chip CH340 has been removed and that of the chip CP1202 has been included. With this, the computer and therefore the Arduino IDE no longer recognizes the chip, the CH340 and the UART clone with the chipCH340 can no longer be used.
This is very difficult but can be overcome by taking some action in LINUX.
To do this, open the LINUX terminal.
$ sudo lsmod
If you are not yet logged in, you will be prompted to do so.
You will now see an overview of all modules and, among other things, the installed UART firmware:
– module
– PL2303 – both USB serial modules
– CH341
If the UART firmware CH340,41 is not visible, enter the following cmd-line text:
$ sudo apt autoremove brltty
This will make – /dev/ttyusb0 – accessible.
Install ESP01-8266 Board in Arduino IDE
To be able to program the ESP01-8266 board under the Arduino IDE, it must first be made known to the IDE. To do this, open the board manager in the Arduino IDE and enter the following – .json – file at additional board managers URLs : http://arduino.esp8266.com/stable/package_esp8266com_index.json
By now selecting the board ESP8266 in the Arduino IDE you can use the ESP01-8266
computer programming.
How the main code works
At the beginning of the sketch, a detailed explanation of the program is given. A number of aspects of the program have already been described in detail above. Still the following:
- the version of the board manager “Generic ESP8266 Module” should not be higher than 3.0.0. A current (higher) version gives error messages in the Arduino IDE for the ESP01 with the UART CD340.
- Check the UART if it contains the chip CH340 or CP2104. If the “old” Chinese clone CH340 is used, you may not be able to get the ESP01 into program mode. (See earlier)
For the readability of the program, it has been decided to include a lot of explanation on the spot of the program part. Furthermore, many lines of program code have been included to monitor the operation of the program in the serial monitor of the Arduino IDE.
To improve the readability of the program, it was also decided to use subroutines as much as possible. These are included in the User Defined Function section.
The basic structure of the program therefore consists of:
– Documentation Section
– Preprocessor Section
– Define section
– Global Declaration
– User Defined Function Section (Subroutines)
– setup
– loop
Documentation Section
Contains the basic principles of the project and general information about the project.
Libraries
Libraries are fragments of code that can be incorporated into your own program. They are program parts written by other enthusiasts and made available for use under GNU license. Among other things, the NTPClient.h library is used. This is a complex piece of software that processes the NTP protocol for you.
Preprocessor Section
Three libraries have been written for this program:
secretswifi.h This program part is included in the main sketch and called when the WiFi connection with the internal LAN network has to be established. It contains the ID and Password of the desired WiFi LAN network. This is done to make it more difficult to pluck and misuse the ID and Password.
indexHTML.h This program component contains Web-Page that becomes visible in the browser. Although it is also possible to include the desired Web-Page in the main program, it has been decided to include this in a separate file and call it when necessary.
notFound_HTML.h This file becomes active when a Web-Page call does not exist, a so-called 404 call.
Definition Section
Fixed or DHCP IP-adress
The definition section of the main sketch starts with setting a Fixed LAN IP-address. Furthermore, the sketch is composed in such a way that it can work in both DHCP- and Fixed-mode by simply putting these Keywords in a string.
IPAddress local_IP(192, 168, 1, 20); IPAddress gateway(192, 168, 1, 1); IPAddress subnet(255, 255, 255, 0); IPAddress DNS1(192, 168, 1, 1); String stringWiFi = "Fixed"; // Take Fixed or DHCP
Setting up NTPClient
Setting up an NTPClient object needs a WiFiUDP object, an NTP server and an offset to specify your time zone. We use pool.ntp.org as the NTP server address. This automatically detects the closest time server from your location.
Lastly, for the UTC offset for your timezone, use this formula: UTC X = X * 60 * 60
For Western Europe Winter time UTC+1 = X = 3600 and for
Summertime UTC+2 = X = 7200
const long utcOffsetInSeconds = 3600;
By default ‘pool.ntp.org‘ is used with 60 seconds update interval and Offset Western European Winter Time = utcOffestInSeconds. You can specify the time server pool and the offset, (in seconds) additionally you can specify the update interval (in milliseconds).
NTPClient timeClient(ntpUDP, “europe.pool.ntp.org”, 3600, 60000);
NTPClient timeClient(ntpUDP, "pool.ntp.org", utcOffsetInSeconds, interval);
WiFi connection
The WiFi connection subroutine starts with the question whether the ESP01 module is in STA(ion) mode or not. If not, it put’s it in Station mode. Then it is checked whether the connection should be DHCP or fixed. The SIDD and Password are obtained from the file secretswifi.h (see later), a bit of security for Over The Air Programming (OTA).
Getting Date and Time
This subroutine retrieves the time and date. The time is obtained directly from getFormattedTime() while the date still needs some editing. The obtained time is stored in the memory location formattedTime and the obtained date in the memory location Date.
Program Files
indexHTML.h
This file is requested in the setup routine with the subroutine void HandleRoot() and builds the WebPage in the browser when it calls the URL of the web page. The index file contains an AJAX script that periodically and Asynchronously requests the date and time in /datenow and /timenow, both subpages of the index page.
The file you can find indexHTML.h.
notfound_HTML.h
If the webpage can’t be found, it’s nice to report it. That is why a separate file notfound_HTML.h has been included for the 404 action.
The file you can find notFound_HTML.h.
secretswifi.h
The ID and Password of the Local WiFi router are included in this file and made available to the main program. The file you can find secretswifi.h
The main sketch
As mentioned earlier, the main program consists of the following parts:
– Documentation Section
– Preprocessor Section
– Define section
– Global Declaration
– User Defined Function Section (Subroutines)
– setup
– loop
The program is provided with a lot of explanation for comprehensibility. Together with the previous explanation and this explanation, it will certainly be easy to follow.
/***************************************************************************************
*
* PA3BYB
*
* ESP01-8266_NTP_Server_OTA-based_AJAX_E.ino
*
****************************************************************************************
*
* Using Board ESP01s-8266 & USB-Serial adapter with CH340 chip FEIYANG
*
* Install board manager "Generic ESP8266 Module" version 3.0.0
* (newer version don't work!!!!)
*
* ESP-01S is a black colored module with 1024k memory. ESP-01S requires only 3.3 volts
* to power up. To flash or upload firmware/sketch, we need an external USB Serial Board
* adapter that supports 3.3 volts. In my case i used the CH340 chip instead of the
* CP2104 chip. This is the reason for using board manager 3.0.0
*
* The serial programming module need to be prepared for programming the the ESP-01S.
* For more information serial board information:
* https://www.diyhobi.com/flash-program-esp-01-using-usb-serial-adapter/
*
*----------------------------------------------------------------------------------------
* The Project Goal
* Realizing date and time on the Web Page whereby both date and time are automatically
* updated.
*----------------------------------------------------------------------------------------
* Project Developments
* - The Web Server presents itself's with the heading tekst:
* “AJAX BASED ESP-01 (8266) NTP Server PA3BYB”
*
* - The IP-address can be FIXED or DHCP.
* - The ID and PSW for WiFi log are included in separate file.
* - The structure of the web page is included in a separate file.
* - Use for the ESP8266 Board version 3.0.0. Necessary for correct use Arduino IDE
* - Format for "WiFi.config(local_IP, gateway, subnet, DNS1)" All four subjects are
* needed.
*----------------------------------------------------------------------------------------
* The sketch use four libraries
* - ESP8266WiFi.h
* - WiFiClient.h
* - ESP8266WebServer.h
*
* - NTPClient.h
* Get time from a NTP server and keep it in sync.
* It can be found https://github.com/arduino-libraries/NTPClient
*
* - WiFiUdp.h
* Creates a named instance of the WiFi UDP class that can send and recieve UDP
* message. On AVR boards, outgoing UDP packets are limited to 72 bytes in size
* currently. For non-AVR boards the limit is 1446 bytes.
*
*---------------------------------------------------------------------------------------
* DATA 2023 02 20 exp. E The current version provides both date and time and is
* updated every second. AJAX is used for updating.
*
****************************************************************************************
* End of Documentation section *
****************************************************************************************
* Preprocessor section *
***************************************************************************************/
#include <ESP8266WiFi.h>
#include <ESP8266WebServer.h>
#include <WiFiClient.h>
#include <NTPClient.h>
#include <WiFiUdp.h>
#include <ArduinoOTA.h>
//-- This file contains the SIDD and Password
#include "secretswifi.h"
// This file include the basic website in HTML
#include "indexHTML.h"
// This file include a simple website for 404
#include "notFound_HTML.h"
/***************************************************************************************
* End of Preprocessor section *
***************************************************************************************
* Define section *
**************************************************************************************/
//---- Set your Static IP adress, Gateway and Subnet
IPAddress local_IP(192, 168, 1, 20);
IPAddress gateway(192, 168, 1, 1);
IPAddress subnet(255, 255, 255, 0);
IPAddress DNS1(192, 168, 1, 1);
//IPAddress primaryDNS(8, 8, 8, 8);
//IPAddress secondaryDNS(8, 8, 4, 4);
//---- Set WiFi connection in DHCP or Fixed IP-Address
String stringWiFi = "Fixed"; // Take Fixed or DHCP
#define OTA_Hostname "ESP01-8266 FIXED IPaddress OTA-based Date & Time"
WiFiUDP ntpUDP; // Set UTP protocoll
/* Setting up an NTPClient object needs a WiFiUDP object, an NTP server and an offset
* to specify your time zone. We use pool.ntp.org as the NTP server address.
* This automatically detects the closest time server from your location. Lastly, for
* the UTC offset for your timezone, use this formula:
* UTC X = X * 60 * 60
* For Western Europe x = 1 * 60 * Winter time UTC+1 = 3600
* x = 2 * 60 * 6060 Summer time UTC+2 = 7200 */
const long utcOffsetInSeconds = 3600;
/* By default 'pool.ntp.org' is used with 60 seconds update interval and Offset
* Western European Winter Time = utcOffestInSeconds
* You can specify the time server pool and the offset, (in seconds) additionally you
* can specify the update interval (in milliseconds).
* NTPClient timeClient(ntpUDP, "europe.pool.ntp.org", 3600, 60000); */
NTPClient timeClient(ntpUDP, "pool.ntp.org", utcOffsetInSeconds);
String formattedTime; // Prepare for formatted Time string
String Time;
String Date; // Prepare for Date string
int Day; // Prepare for day integer
int Month; // Prepare for Month integer
int Year; // Prepare for Year integer
/***************************************************************************************
* End of Define section *
***************************************************************************************
* Global Declaration *
**************************************************************************************/
//--------------------------- Start the Web Server -------------------------------------
ESP8266WebServer server(80); // Start webserver on poort 80
/***************************************************************************************
* User Defined Function section *
* Subroutines *
**************************************************************************************/
//------------------------------ WiFi Connection ---------------------------------------
void WiFiConnect(){
if(WiFi.getMode() != WIFI_STA){ // Check WiFi mode if not STA
WiFi.mode(WIFI_STA); //.... make it STA
delay(1000);
}
if (stringWiFi == "Fixed"){ // Configure static IP address
// The WiFi.config() need all four subjects otherwise you get trouble reading
// Correct Time & Date
WiFi.config(local_IP, gateway, subnet, DNS1); // DNS must be filled in
Serial.print("\n Connected to fixed IP ");
} else {
Serial.println("Connected to DHCP IP");
}
delay(500);
Serial.print("\n Start WiFi connection.");
delay(500);
WiFi.begin(ssidsta,passwordsta); // Try to connect WiFi router
while(WiFi.status()!= WL_CONNECTED){ // Connect to WiFi
delay(500);
Serial.print(".");
}
//---------------- print the SIDD of the network you're attached to ----------
if(WiFi.status() == WL_CONNECTED) {
Serial.print("\n Connect to SSID: "); // Connected to SSID ..
Serial.print(WiFi.SSID());
}
//---------------- print your WiFi shield's IP address -----------------------
Serial.print("\n IP adress : "); // Connect to IP-adress
Serial.println(WiFi.localIP());
Serial.println("\n WiFi connected.");
//--------------- print the recieved signal strenght -------------------------
long rssi = WiFi.RSSI();
Serial.print("\n WiFi signal strenght (RSSI): ");
Serial.print(rssi);
Serial.println(" dBm");
}
//-------------------------- Start OTA-server --------------------------------------
void StartOtaServer(){
ArduinoOTA.setHostname(OTA_Hostname);
Serial.print("\n Start OTA with hostname: ");
Serial.println(OTA_Hostname);
ArduinoOTA.begin();
Serial.print("\n OTA Ready");
}
//-------------------------- Get Time & Date ----------------------------------------
/*Inside the handle_OnConnect() function are the commands that fetch the current date
and time readings from their respective libraries. Using the time structure we set
earlier, we get the current date. Whereas for time, we use getFormattedTime() directly
from the NTPClient library. Finally, server.send() returns the data to the client. */
void handle_OnConnect() {
timeClient.update();
/* The epoch time function returns the number of seconds that have elapsed since
January 1, 1970. We use this function along with a time structure to get the date.*/
time_t epochTime = timeClient.getEpochTime();
int currentHour = timeClient.getHours();
int currentMinutes = timeClient.getMinutes();
int currentSeconds = timeClient.getSeconds();
/* Struct is similar to class but for numeric(value type) and is a kind of template
* with executable code
* gmtime = Greenwith Mean Time
* pmt = present time
* tm = is de struct name en bevat de tijd in sec, min, hour, mday, mon (0-11),
* year -1900), wday, yday, isdts
* time_t = pointer van het type time_t dat de time value bevat (absolute time) */
struct tm *ptm = gmtime ((time_t *)&epochTime);
int monthDay = ptm->tm_mday;// Serial.print (" Month Day :");
int currentMonth = ptm->tm_mon+1;
int currentYear= ptm->tm_year+1900;
formattedTime = timeClient.getFormattedTime();
// Serial.print("\n formatted time 1: ");
// Serial.println(formattedTime);
Date = String(monthDay) + "-" + String(currentMonth) + "-" + String(currentYear) ;
Serial.println();
Serial.print("\n Date = ");
Serial.println( Date);
Time = String(currentHour) + ":" + String(currentMinutes) + ":" +String(currentSeconds);
Serial.print(" Time = ");
Serial.println( Time);
server.send(200, "text/plain", formattedTime);
Serial.print("\n Formatted Time = ");
Serial.println(formattedTime);
}
//--------------------------- handle Date -----------------------------------------
void handle_Date(){
server.send(200, "text/plain", Date);
Serial.print("\n Date today is ");
Serial.println(Date);
}
//------------------------ Build the Web Page --------------------------------------
void HandleRoot() {
server.send(200, "text/html", indexHTML);
}
//------------------------ Web Page not Found --------------------------------------
void HandleNotFound(){
server.send(404, "text/html", notFound_HTML);
}
/*************************** End of Subroutines ***********************************/
/*************************** Start Main Program ***********************************/
//------------------------------- SetUp ------------------------------------------
void setup() {
Serial.begin(115200);
delay(500); // Wait for Serial.begin
WiFiConnect(); // Connect to WiFi
StartOtaServer(); // Start OTA server
server.on("/", HandleRoot);
server.on("/timenow", handle_OnConnect);
server.on("/datenow", handle_Date);
server.onNotFound(HandleNotFound);
server.begin();
}
//-------------------------------- LOOP -------------------------------------------/
void loop() {
ArduinoOTA.handle();
server.handleClient();
}