Wiki source code of Data-logging and Supervisory Control

Last modified by Matej Durcik on 2022/05/17 14:45

author	version	line-number	content
		1	= Overview =
		2
		3	The LEO Data-logging and Supervisory Control (DSC, also known as SCADA and C&D) is separated into two systems:
		4
		5	1. Landscape monitoring and control - This is currently distributed over 14 National Instruments compact re-configurable input/output (cRIO) real-time computers. ([[http:~~/~~/www.ni.com/compactrio/>>url:http://www.ni.com/compactrio/\|\|target="_blank"]] ) The NI cRIO allows for a variety of I/O modules and is equipped with an onboard Field Programmable Gate Array (FPGA) allowing for additional signal processing and custom digital communication protocols.
		6	1. LEO biome environmental monitoring and control - Monitored and controlled by a variety of Allen Bradley PLCs. Details on this to come.
		7
		8	(((
		9	The first part, landscape monitoring and irrigation control, was designed to provide control and data for scientific use. It was designed with two major goals:
		10
		11	1. To be robust and minimize data loss.
		12	1. Allow for easy and seamless integration of new sensors and new data acquisition hardware
		13
		14	(((
		15	In order to achieve these goals the system was designed to be very loosely coupled. Each subsystem is required to perform an assigned task with minimal dependence on other subsystems. The database logging system was completely decoupled from the data acquisition devices, requiring data to be provided as appropriately formatted ASCII text files. This allows new acquisition devices to be added with no impact on other acquisition devices and (ideally) only configuration changes to the database loader.
		16	)))
		17	)))
		18
		19	A basic diagram of data acquisition flow is shown below:
		20	[[image:Data-Flow-Diagram_for_LEO.png\|\|style="display:block; margin-left:auto; margin-right:auto"]]
		21
		22	== Notes ==
		23
		24	* [[Measurement file format>>doc:MeasurementFile]]
		25	* [[Calculation notes>>doc:CalculationNotes]]
		26
		27	== [[Hardware>>doc:DSC_Hardware]] ==
		28
		29	{{box}}
		30	{{display reference="DSC_Hardware"/}}
		31	{{/box}}
		32
		33	== [[Software>>doc:DSC_Software]] ==
		34
		35	{{box}}
		36	{{display reference="DSC_Software"/}}
		37	{{/box}}
		38
		39	== Using the LEO data system ==
		40
		41	Uploading data to the Data Gateway is easy and requires only three things: Sensor meta data configured in the database, a correctly formatted ASCII text file, and FTP capability.
		42	Step 1) Create the sensor in the LEO database and configure the meta data. The sensor must exist with appropriate variables, calibrations, etc. within the database in order for data for a particular sensor to be logged.
		43	Step 2) Record data the LEO way! Data is presented in tab delimited ASCII text files called "Measurement Files" (measurement files are described here). Using this formatting scheme allows the data gateway to parse the file and insert data into the database. Plus, it's easy to read and debug!
		44	Step 3) Upload the measurement file to the Data Gateway via FTP. FTP is a simple and well established protocol. Once a file is uploaded to the server, it is ready to be processed!
		45	\\Q: Why not log directly to the database?
		46	A: Since the system is distributed there are many remote components. This scheme limits the number of devices writing to the database to only one. This allows changes to be made to the database with minimal impact on the data acquisition devices. If the database goes offline (unexpectedly or planned) the measurement files will still be stored on the gateway server and not be processed. Thus, data is not lost during database outages.
		47	Q: Why use measurement files?
		48	A: This stems from the philosophy that data loggers should be self reliant and write data locally. This reduces the dependence on a stable network connection and ensures data integrity during server outages.
		49	Q: Why use FTP?
		50	A: FTP is simple and easy to use. Many software packages and programming languages have the capability to use FTP; Therefore, it can be easy to automate the uploading measurement files to the server.