ECS Build/Design Document
Build ECS from sources (for Windows/Linux64/Linux32/ARM):
PC Setup
Build
Design notes
Files
Notes:
This file was downloaded from https://docs.microsoft.com/en-us/windows/wsl/install-manual.
Debian can also be installed via the Windows Store, but I do not recommend it,
as it can install a later version of Debian (which causes the GLIBC problem described below).
I chose Debian (vs Ubuntu) for its stability/simplicity: e.g. all cross-compiler packages install
without having to customize the repository list (/etc/apt/sources.list).
If ECSSources.7z has been updated since the last build:
Build distros: allib.bat (run in C:\Mark)
Resulting distros:
Omnipotence (mark@OmnipotenceSoftware.com / 423-368-2339):
ECS_SETUP.exe (ECS/Windows)
ecs.tar.gz (ECS/Linux)
ALL files created/maintained are standard ASCII text files (vs binary files, which can't be easily inspected by a person).
ECS uses char[] arrays/pointers inst of std::string - which proved significantly faster per *past* benchmarks.
But recent benchmarks show that speeds are comparable (due to std::string class enhanements).
std::strings can more convenient - esp as they support operators: e.g. st += "12345" vs st = AppSt(st, "12345");
Regardless, the exclusive use of char[] allows us to use a *single* data type for all byte-array data (text strings, file buffers, image data, etc).
It also allow us to monitor memory usage (as all char[] pointers are allocated/freed via *our* rtns - e.g. AllocU1, FreeSP).
ECS is "single threaded" but for network connection requests (which are done via ascynrounous threads).
The main advantage to this approach is *predictabilty* (if a test fails, it will fail the same way ever time).
The disadvantage is that object service rtns must do what work can be done *at the moment*,
save any context info related to incomplete/in-progess tasks (so they can be resumed on the next call),
and return control to ECS asap.
Web pages (generated by the ECS web server) contain javascript code which communicates with ECS via websockets
This code is defined in sections "part0_js" to "part17_js" (in ECS.c).
I had to break it up into sections due to the C++ limitation on the size of text strings.
I've dedicated many hours tweaking/benchmarking the memory allocation, file I/O, and text search/replace functions.
I believe them to be as fast as possible.
Naming conventions (in rtn/arg names):
Bf Buffer: char[] array or pointer
St String: char[] array of text (terminated by NULL)
SP String Pointer: Allocated char array of text (terminated by NULL)
Memory Allocation:
All memory allocation/free operations are done by my functions (AllocU1/FreeSP/ReplaceSP in mem_lb.c) ("SP" is String Pointer).
They use malloc()/free() functions (vs "new" / "delete" operators).
Routing all memory allocation tasks thru these rtns allows us to track the amount of memory allocated (persistent and dynamic) and detect memory leaks.
AllocU1() allocates a structure ("MEMDSC") that lies before the pointer returned.
This structure defines the allocated size, a dynamic flag (vs persistent), and an optional index into the LPMA pointer array (if applicable - see below).
FreeSP/ReplaceSP access this structure by simply subtracting the size of MEMDSC from the passed pointer.
Loop memory denotes temporary buffers (as returned by various functions) that are freed at the top of the main pgm loop (among other places):
Loop Memory / Internal: LPMI buffers are defined in a static/fixed-length array (super fast). These are used for limited-size buffers when available.
Loop Memory / Allocated: LPMA pointers are stored in an array. This is for buffers that are too large for LPMI (or if no LPMI buffers are available)
Internal Transitory Memory: ITM buffers are defined in a static/fixed-length array (like LPMI). These are used for temporary buffers that will be freed at some point (such as FIFO data)
Z Lists:
ECS uses "Z Lists" (ZL) extensively.
These allow *very* large/flexible lists to be defined with minimal data - e.g.:
static U1 *zl_misc_choices[] = {
"Off, On, Unknown", // Off, On, Unknown
"0...5", // 0, 1, 2, 3, 4, 5
"0...10+2", // 0, 2, 4, 6, 8, 10 ("+2" denotes an increment of 2)
"0.0...0.5", // 0.0, 0.1, 0.2, 0.3, 0.4, 0.5
"Sunday...Saturday", // Sunday, Monday, . . . Saturday
"1/1...12/31", // 1/1, 1/2, . . . 12/31
"12:00 AM...12:05 AM", // 12:00 AM, 12:01 AM, 12:02 AM, 12:03 AM, 12:04 AM, 12:05 AM
"12:00:00 AM...12:00:05 AM", // 12:00:00 AM, 12:00:01 AM, 12:00:02 AM, 12:00:03 AM, 12:00:04 AM, 12:00:05 AM
"00:00...00:05/TMC", // minute-counts: 00:00, 00:01, 00:02, 00:03, 00:04, 00:05
"00:00:00...00:00:05/TMC", // second-counts: 00:00:00, 00:00:01, 00:00:02, 00:00:03, 00:00:04, 00:00:05
0};
The primary VLST rtns are:
Logic flow:
start init.htm in the default browser (the page that says "ECS initializing (please wait) . . .")
this page refreshes itself every 5 seconds
read main.cfg and construct the objects therein
re-create init.htm such that it displays the ECS "Home" page
init.htm will subsequently refresh itself and display "Home"
the javascript therein will then connect to ECS (via a websocket), and "subscribe" to the Properties in the page.
it subsequently requests updates (e.g. Property value changes) every 1/2 second
do until exit
service all objects (incl scripts)
service all sockets (respond to http web page requests, and update requests from websockets)
Files:
NOTE: All .C sources are actually C++ (whose convential extension is ".CPP").
| File | Description |
| \Program Files\Microsoft Visual Studio\2022\Community\VC\Redist\MSVC\14.30.30704\x86\Microsoft.VC143.CRT\msvcp140.dll | Visual C++ Redistributable for Visual Studio 2015 (www.microsoft.com/en-us/download/details.aspx?id=48145) |
| \Program Files\Microsoft Visual Studio\2022\Community\VC\Redist\MSVC\14.30.30704\x86\Microsoft.VC143.CRT\vcruntime140.dll | Visual C++ Redistributable for Visual Studio 2015 (www.microsoft.com/en-us/download/details.aspx?id=48145) |
| \Program Files (x86)\Windows Kits\10\Lib\10.0.19041.0\um\x64\Xinput9_1_0.lib | for XBox360Controller Class (Windows 10 SDK, version 2004 (10.0.19041.0) developer.microsoft.com/en-us/windows/downloads/sdk-archive |
| \installs\ARMLibs\* | for Linux/ARM: Numerous libs compiled on Raspberry PI |
| \installs\BashOnWindows\DebianGNULinux_1-1-3-0_x64__76v4gfsz19hv4.Appx | Debian OS package (that runs *IN* Windows - so we can build Windows *and* Linux versions in Windows) |
| \Installs\GNUPlot\gp527-win32-mingw\gnuplot\bin\*.dll | for DataPlot Class |
| \installs\GNUPlot\gp527-win32-mingw\gnuplot\bin\gnuplot.exe | for DataPlot Class |
| \installs\Inno (Win software installer)\* | creates Windows ECS installer (ECS_SETUP.exe) |
| \Program Files (x86)\GnuWin32\bin\jpeg62.dll | for Windows Image.Compare and "internal" camera motion detection (i.e. done manually by ECS) |
| \installs\jpegs\ARM\libjpeg.a | for Linux/ARM Image.Compare and "internal" camera motion detection (i.e. done manually by ECS) |
| \installs\jpegs\jpeg-6b\libjpeg.a | for Linux/32 Image.Compare and "internal" camera motion detection (i.e. done manually by ECS) |
| \installs\MySQL\* | Windows/Linux MySQL installers (for MySQLDataBase Class) |
| \Program Files\MySQL\MySQL Connector C++ 8.0\lib64\libssl-1_1-x64.dll | dev.mysql.com/downloads/connector/cpp/ |
| \Program Files (x86)\MySQL\MySQL Connector C++ 8.0\lib\mysqlcppconn-9-vs14.dll | MySQL Connector/C++ 8.0 dll (dev.mysql.com/doc/connector-cpp/8.0/en/) |
| \Program Files (x86)\MySQL\MySQL Connector C++ 8.0\lib\libcrypto-1_1.dll | MySQL Connector/C++ 8.0 dll (dev.mysql.com/doc/connector-cpp/8.0/en/) |
| \Installs\canvas-gauges-master\* | for Gauge3 Widgets |
| \Installs\ImageMagick\identify.exe | used to determine image file height/width (imagemagick.org/script/download.php) |
| \Installs\mailsend\mailsend.exe | for EMail Class (default emailer for windows) (sourceforge.net/projects/mailsend.mirror) |
| \Installs\Postie (emailer)\postie.exe | for EMail Class (alternate emailer for Windows) (postie.apponic.com) |
| \Installs\SNMP\snmpget.exe | for SNMPObject Class |
| \Installs\SNMP\snmpset.exe | for SNMPObject Class |
| \Installs\SQLite\sqlite3.exe | for SQLiteDataBase Class |
| \installs\OpenSSL\openssl-1.1.1\Include | for HTTPS support |
| \installs\OpenSSL\openssl-master\include | for HTTPS support |
| \installs\OpenSSL\openssl-master\libcrypto-3.dll | for HTTPS support |
| \installs\OpenSSL\openssl-master\libssl-3.dll | for HTTPS support |
| \mark\*.gui | custom Class GUIs (e.g. weather.gui) |
| \mark\*.widget* | widget defs |
| \mark\*ModBusDeviceMap.txt | modbus device register maps |
| \mark\ActivateWindow.c | used by ActivateWindow() to bring window to foreground (when running as daemon) |
| \mark\ActivityLogModifiers.txt | denotes the Classes that are to trigger Activity logging when THEY make a change |
| \mark\ActivityLogProperties.txt | denotes the Properties that are to trigger Activity logging when they are modified |
| \mark\allb.bat | build all libraries and pgms (windows & linux) |
| \mark\array_lb.c | array/symbol rtns |
| \mark\bacrp | reads BACNET value (linux) |
| \mark\bacrp.exe | reads BACNET value (windows) |
| \mark\bacwp | writes BACNET value (linux) |
| \mark\bacwp.exe | writes BACNET value (windows) |
| \mark\base64_lb.c | BASE64 encode/decode rtns (for images and passwords to/from web pages) |
| \mark\cc.bat | compile C++ source (windows) |
| \mark\cc.sh | compile C++ source (linux/64) |
| \mark\cc32.sh | compile C++ source (linux/32) |
| \mark\ccARM.sh | compile C++ source (linux/ARM) |
| \mark\ccl.bat | compile/link C++ source (windows) |
| \mark\ccl.sh | compile/link C++ source (linux/64) |
| \mark\ccl32.sh | compile/link C++ source (linux/32) |
| \mark\cclall.bat | compile/link C++ source (windows & linux) |
| \mark\cclARM.sh | compile/link C++ source (linux/ARM) |
| \mark\click.mp3 | click sound (for use in web pages) |
| \mark\clipboard_lb.c | Clipboard rtns |
| \mark\colors.htm | hex color map |
| \mark\com_lb.c | "COM" (serial) I/O rtns |
| \mark\console_lb.c | Console (e.g. keyboard/terminal) I/O rtns |
| \mark\conversions_lb.c | Conversion rtns (integer to text, etc) |
| \mark\cr.bat | compile and replace in ut.lib library (windows) |
| \mark\cr.sh | compile and replace in ut.a library (linux/64) |
| \mark\cr32.sh | compile and replace in ut32.a library (linux/32) |
| \mark\crARM.sh | compile and replace in utARM.a library (linux/ARM) |
| \mark\ecs.c | "main" source (also includes javascript data/functions defined via text strings) |
| \mark\ECS.ISS | INNO (windows) install file |
| \mark\ecs.manifest | ?? |
| \mark\ecs.service | Linux service file (64 bit) |
| \mark\ecs32.service | Linux service file (32 bit) |
| \mark\ECSAPI.c | Reads or writes an ECS Property via API command (command-line pgm) |
| \mark\ecsARM.service | Linux service file (ARM) |
| \mark\ecsb.bat | build all executables (windows & linux) |
| \mark\ECSBuildAndDesignNotes.htm | This build/design document |
| \mark\ECSErrTray.ico | windows pgm tray icon (when new errors have been logged) |
| \mark\ECSFavicon.ico | pgm icon (appears in browser tab). Copied to favicon.ico on install |
| \mark\ECSIB.bat | build windows/linux installations (ECS_SETUP.exe / ecs.tar.gz) |
| \mark\ECSLinuxZp.sh | build linux install (ecs.tar.gz) |
| \mark\ECSLogo.png | company logo |
| \mark\ECSTray.ico | windows pgm tray icon (when no new errors have been logged) |
| \mark\edm.bin | mark's text-file editor (linux64) |
| \mark\edm.exe | mark's text-file editor (windows) |
| \mark\edm32.bin | mark's text-file editor (linux32) |
| \mark\edmARM.bin | mark's text-file editor (linuxARM) |
| \mark\Elkm1Events.txt | for ELK-M1 Classes |
| \mark\error.mp3 | error sound file (played in browser) |
| \mark\exec_lb.c | rtns for executing/monitoring external processes |
| \mark\ExternalLogProperties.txt | denotes Properties that are considered "external" to PC. All changes to such Properties are logged to ExternalChanges.log |
| \mark\ffmpeg.exe | used to convert camera images to videos (ffmpeg.org/download.html) |
| \mark\ffplay.exe | used by FFPlayFile() to play audio files (ffmpeg.org/download.html) |
| \mark\field_lb.c | input "field" rtns (used only by mark's pgms - NOT used by ECS) |
| \mark\file_lb.c | File I/O rtns (incl MRF:Memory-Resident-Files) |
| \mark\fnd_lb.c | find(search) / string match / parsing rtns |
| \mark\ftpx.c | ftp upload/download/delete pgm (used by FTPClient Class) |
| \mark\HeartbeatMonitor.c | monitors a process "heartbeat" and restarts it no heartbeat detected after n seconds |
| \mark\holidays.txt | US holidays (used by Clock.Date Property) |
| \mark\HubitatDriverCapabilities.txt | used by HubitatDevice Class |
| \mark\InstallPackages.sh | installs packages used by ecs |
| \mark\InstallWindowsDebianPackages.sh | installs packages (in Debian Windows Subsystem for Linux) used to build ECS/Linux in windows |
| \mark\jpeg_lb.c | jpeg rtns |
| \mark\justgage.js | Gauge2 widget |
| \mark\kb_lb.c | Keyboard I/O rtns |
| \mark\lm.exe | mark's directory list pgm (similar to "dir" and "ls") |
| \mark\lnk.bat | link a pgm (windows) |
| \mark\lnk.sh | link a pgm (linux/64) |
| \mark\lnk32.sh | link a pgm (linux/32) |
| \mark\lnkARM.sh | link a pgm (linux/ARM) |
| \mark\log_lb.c | log/error-processing rtns |
| \mark\main.cfg | default ECS configuration |
| \mark\manual.htm | ECS users manual |
| \mark\mem_lb.c | memory/etc rtns |
| \mark\misc_dt.c | misc/global variables/arrays |
| \mark\MobileAgents.txt | mobile http agents+info |
| \mark\modbus.h | modbus packet structures |
| \mark\mp3_lb.c | mp3 rtns (used only by mark's pgms - NOT used by ECS) |
| \mark\mysql-connector-c++-8.0.29-src\* | MySQL includes (for windows build) |
| \mark\mysqlcppconn-9-vs14.lib | MySQL library (for windows build) |
| \mark\network_lb.c | network (ethernet, TCP/IP) rtns |
| \mark\NOAAAlertZones.txt | NOAA Alert Zones (used by Weather Class) |
| \mark\notice.htm | user notice (used in main.cfg) |
| \mark\os_vsn.c | OS info rtn |
| \mark\pausex.exe | pause pgm for use in batch files: arg1=prompt, arg2=timeout (in seconds) |
| \mark\PrintFileNameIfNewerThanOthers.bin | used by cc*.sh (linux) to ignore compile if object is newer than source |
| \mark\PrintFileNameIfNewerThanOthers.exe | used by cc*.bat (windows) to ignore compile if object is newer than source |
| \mark\ProblemReport.htm | problem report form |
| \mark\PropertyHelp.c | creates Property Help (hover text) for every Property (as extracted from manual.htm) |
| \mark\PropertyHelp.txt | Property Help (hover text) for every Property |
| \mark\raphael-2.1.4.min.js | used by Dial2*.widget and Gauge2*.widget |
| \mark\reboot.bat | used by Program Class to reboot PC (when set to "Reboot") |
| \mark\reboot.sh | used by Program Class to reboot PC (when set to "Reboot") |
| \mark\ReplaceTextInFile.c | replace text in a file |
| \mark\ReplaceTextInFile.exe | replace text in a file |
| \mark\server.crt | for HTTPS support |
| \mark\server.key | for HTTPS support |
| \mark\SimKey.c | Emulates keystroke or mouse activity (windows). Used by Window.Key Property |
| \mark\snmp_lb.c | SNMP (Simple Network Management Protocol) rtns |
| \mark\snmp_trap_linux.cfg | Example cfg for trapping SNMP in Linux |
| \mark\sound_lb.c | Sound-related rtns |
| \mark\sqlite3.c | SQLite3 library. Used by SQLDataBase Class (sqlite.org) |
| \mark\sqlite3.h | SQLite3 library header. Used by SQLDataBase Class (sqlite.org) |
| \mark\sql_lb.c | SQLite3/MySQL rtns |
| \mark\stdout_capture.c | Used by CC.BAT/etc to capture compile output and check for errors |
| \mark\stdout_capture.exe | Used by CC.BAT/etc to capture compile output and check for errors |
| \mark\styles.css | CSS styles used by ECS pages |
| \mark\sun_calc.c | Used to calc sun rise/set times per latitude/longitude/zone/year/month/mday |
| \mark\sun_calc2.c | Used to calc dawn/dusk times per latitude/longitude |
| \mark\sys.h | Misc constants |
| \mark\time_lb.c | Time-related rtns |
| \mark\tts_lb.c | Text-To-Speech rtns |
| \mark\tts_lexicon.txt | Denotes text/words which must be "re-spelled" in order to be pronounced properly by the text-to-speech. Also expands some abbreviations (e.g. TX -> Texas) |
| \mark\update.sh | Used in updating ECS |
| \mark\ut.h | Global rtn/data declarations |
| \mark\ut32b.sh | build ut.lib (linux/32) |
| \mark\utARMb.sh | build ut.lib (linux/ARM) |
| \mark\utb.bat | build ut.lib (windows) |
| \mark\utb.sh | build ut.lib (linux/64) |
| \mark\ValueListIgnore.txt | Denotes which Classes/Names/Properties/values are to be omitted from the ValueList file (used by Program.ValueList) |
| \mark\vlc_lb.c | VLC (media player) rtns |
| \mark\vlst_lb.c | VLST (Virtual List) rtns (used extensively by ECS) |
| \mark\W800.btn | W800 buttons (used by W800 Class) |
| \mark\W800.chc | W800 values (used by W800 Class) |
| \mark\webcam_lb.c | WebCam rtns |
| \mark\webdl.c | WWW file download pgm |
| \mark\Widget*Decl.txt | Widget declarations |
| \mark\WidgetDefaults.txt | Denotes default widget for various Properties |
| \mark\WidgetHelp.png | Image showing various widgets/formats |
| \mark\WidgetHelp.htm | HTML showing WidgetHelp.png |
| \mark\WidgetImages\* | Widget images (bulbs, floodlights, switches, etc) |
| \mark\win_lb.c | Windows-specific rtns |
| \mark\ZipcodeInfo.txt | US zip codes info (used by Clock and Weather Classes) |
| \mark\ZipCodes.chc | US zip codes (used by Clock and Weather Classes) |