ecology - User contributions [en]

ECMWF

2016-03-02T16:04:50Z

Judy: /* Helpful Functions & Queries */

== ECMWF ==

Datamodel for the tables in the ECMWF schema.

Data is created by the ECMWF and retrieved from the KNMI ftp site as GRIB files. The original GRIB files can also be retrieved from the flysafe server from :

[https://flysafe-pps.grid.sara.nl/archive/knmi/ecmwf/ ECMWF Data Archive]

More information about the parameters in these GRIB files can be retrieved from the [http://www.ecmwf.int/services/dissemination/3.1/Products_in_GRIB_code.html ECMWF website].

== INVARIABLE ==

The INVARIABLE table contains data that is invariant over time.

{|border=1
|'''LATITUDE''' || degrees * 100 to make this a integer
|-
|'''LONGITUDE''' || degrees * 100 to make this a integer
|-
|Z || 129 Z Geopotential surface/model orography
|-
|LSM || 172 LSM Land sea mask
|-
|LOCATION || Latitude/longitude as GIS location
|}

The GIS LOCATION is only added to this table. This is because always the same values for latitude and longitude are used and then if the GIS location is required for the ANALYSIS or FORECAST tables it can be retrieved by joining to this table on the LATITUDE, LONGUTUDE fields.

== FORECAST and UVA_FORECAST ==

The FORECAST table contains predicted values in steps of 3 hours till 72 hours ahead. The MODELTS is the timestamp for which this prediction applies. Two times a day new Forecast GRIB files will arrive. The newer FORECAST data that arrives for the same timestamp will overwrite the older data.

The FORECAST (and UVA_FORECAST) table itself is a master table that does not contain any data. The inherited tables
FORECAST<YEAR><MONTH> contain the data, but the master FORECAST or UVA_FORECAST table can be used to SELECT the data.

Data till 2009-09 is owned by the UvA and can be accessed by anybody with ECMWF_READ rights by using the FORECAST_UVA table. The FORECAST table itself also references data after 2009-09 that is owned by RNLAF for which special access rights ECMWF_RNLAF_READ are required.

{|border=1
|'''MODELTS'''||Time for which this forecast applies = model run + step hours
|-
|'''LATITUDE'''||degrees * 100 to make this a integer
|-
|'''LONGITUDE'''||degrees * 100 to make this a integer
|-
|STEP||model step (smaller is more recent forecast)
|-
|SSRD||169 SSRD surface solar radiation downwards accumulated
|-
|TP||228 TP total precipitation accumulated
|-
|SP||134 SP surface pressure
|-
|BLH||159 BLH boundary layer height
|-
|TCC||164 TCC total cloud coverage
|-
|U10||165 10U 10 meters wind U-component m/s
|-
|V10||166 10V 10 meters wind V-component m/s
|-
|T2||167 2T 2 metres air temperature
|-
|D2||168 2D 2 metres dew point temperature
|-
|LCC||186 LCC low cloud coverage
|-
|''Pressure level 1000''||
|-
|T_1000||130 T temperature
|-
|U_1000||131 U U-wind component
|-
|V_1000||132 V V-wind component
|-
|GH_1000||156 GH geopotential height
|-
|R_1000||157 R relative humidity
|-
|''Pressure level 925''||
|-
|T_925 real||130 T temperature
|-
|U_925 real||131 U U-wind component
|-
|V_925 real||132 V V-wind component
|-
|GH_925 real||156 GH geopotential height
|-
|R_925 real||157 R relative humidity
|-
|''Pressure level 850''||
|-
|T_850||130 T temperature
|-
|U_850 real||131 U U-wind component
|-
|V_850 real||132 V V-wind component
|-
|GH_850 real||156 GH geopotential height
|-
|R_850 real||157 R relative humidity
|-
|''Pressure level 700''||
|-
|T_700 real||130 T temperature
|-
|U_700 real||131 U U-wind component
|-
|V_700 real||132 V V-wind component
|-
|GH_700 real||156 GH geopotential height
|-
|R_700 real||157 R relative humidity
|}

== ANALYSIS and UVA_ANALYSIS ==

The ANALYSIS and UVA_ANALYSIS table contains values from the Analysis GRIB files at 18, 00, 06 and 12 UTC

Data till 2009-09 is owned by the UvA and can be accessed by anybody with ECMWF_READ rights by using the ANALYSIS_UVA table. The ANALYSIS table itself also references data after 2009-09 that is owned by RNLAF for which special access rights ECMWF_RNLAF_READ are required.

{|border=1
|'''MODELTS'''||Time for which this forecast applies = model run
|-
|'''LATITUDE'''||degrees * 100 to make this a integer
|-
|'''LONGITUDE'''||degrees * 100 to make this a integer
|-
|SP||134 SP surface pressure
|-
|TCC||164 TCC total cloud coverage
|-
|U10||165 10U 10 meters wind U-component
|-
|V10||166 10V 10 meters wind V-component
|-
|T2||167 2T 2 metres air temperature
|-
|D2||168 2D 2 metres dew point temperature
|-
|LCC||186 LCC low cloud coverage
|-
|''Pressure level 1000''||
|-
|T_1000||130 T temperature
|-
|U_1000||131 U U-wind component
|-
|V_1000||132 V V-wind component
|-
|GH_1000||156 GH geopotential height
|-
|R_1000||157 R relative humidity
|-
|Pressure level 925||
|-
|T_925||130 T temperature
|-
|U_925||131 U U-wind component
|-
|V_925||132 V V-wind component
|-
|GH_925||156 GH geopotential height
|-
|R_925||157 R relative humidity
|-
|Pressure level 850||
|-
|T_850||130 T temperature
|-
|U_850||131 U U-wind component
|-
|V_850||132 V V-wind component
|-
|GH_850||156 GH geopotential height
|-
|R_850||157 R relative humidity
|-
|Pressure level 700||
|-
|T_700||130 T temperature
|-
|U_700||131 U U-wind component
|-
|V_700||132 V V-wind component
|-
|GH_700||156 GH geopotential height
|-
|R_700||157 R relative humidity
|}

== COMBINED ==

If you want to use Analysis data if available for a timestimp but otherwise Forecast, the COMBINED VIEW can be used. If a ANALYSIS value is available for a timestamp and latitude/longitude the ANALYSIS value will be used, otherwise the FORECAST value is given.

== Historical data ==

In addition ongoing process of adding ECMWF data since November 11 2007, a historical data set has been loaded. That historical dataset is stored in additional inherited tables FORECASTHISTORY and
ANALYSISHISTORY which inherit from FORECAST and ANALYSIS respectively.

The FORECASTHISTORY does not always contain TP (228 TP total precipitation accumulated). Instead of this it contains the following values :

{|border=1
|LSP||142 LSP Large scale precipitation
|-
|CP||143 CP Convective precipitation
|}

The historical dat sets span the following periods and locations

{|border=1
|'''Start'''||'''End'''||'''Area'''||'''Grid'''
|-
|2007-01-01||2007-11-15||62degN 10degW 335degN 20degE||0.25x0.25deg
|-
|2003-02-15||2003-06-15||62degN 10degW 335degN 20degE||0.5x0.5deg
|-
|2005-01-01||2006-12-31||62degN 10degW 335degN 20degE||0.5x0.5deg
|}

== Helpful Functions & Queries ==

The following functions have been defined in Postgres to select the nearest gridpoint in spacetime for the ECMWF analysis and forecast tables.

These are '''ecmwf.forecast_nearest''' and '''ecmwf.analysis_nearest''' and they take a latitude, longitude and timestamp as parameter.

They can be used in the following way, which will return all the fields in the relevant ECMWF data:

SELECT * FROM ecmwf.forecast_nearest( 52.35983, 5.615, timestamp '2008-05-19 13:34:00');

SELECT * FROM ecmwf.analysis_nearest( 52.35983, 5.615, timestamp '2008-05-19 13:34:00');

Note that the model latitude and model longitude are returned as integers and multiples of 100. To return latititude and longitude as a decimal (for example to compare to the latitude and longitude of your GPS data) you need to first convert these fields to decimals:

SELECT f.latitude/100::decimal, f.longitude/100::decimal
FROM ecmwf.forecast_nearest( 52.35983, 5.615, timestamp '2008-05-19 13:34:00') f;

These function can for example be used in a join as follows :

SELECT g.device_info_serial
, g.date_time
, g.latitude
, g.longitude
, g.speed_2d
, g.altitude
, (ecmwf.forecast_nearest( g.latitude, g.longitude,g.date_time)).*
FROM gps.ee_tracking_speed_limited g
WHERE g.device_info_serial = 1
AND g.latitude <> 0
AND g.userflag = 0
ORDER BY g.date_time;

Although these functions were converted to PLPgSQL to speed them up, a faster ways to retrieve the same data is with (thanks to Floris Sluiters):

SELECT device_info_serial, date_time, latitude, longitude, speed_2d, altitude, (x::ecmwf.forecast).*
FROM (SELECT
g.*
,((SELECT f
FROM ecmwf.forecast f
WHERE f.latitude = round(g.latitude * 100 / 25)::integer * 25
AND f.longitude = round(g.longitude * 100 / 25)::integer * 25
AND f.modelts = TIMESTAMP 'epoch' + round(EXTRACT(EPOCH FROM g.date_time)
/ (60*60*3))::integer * INTERVAL '3 hours' limit 1)::ecmwf.forecast) as x
FROM gps.ee_tracking_speed_limited g
WHERE g.device_info_serial = 1
AND g.latitude <> 0
AND g.userflag = 0
ORDER BY g.date_time) t

Below is example query that retrieves data for a model grid point closest to user provided coordinates and for one model time stamp (eg. 12:00) for a series of consecutive days

SELECT f.modelts, f.latitude, f.longitude, f.blh
FROM ecmwf.forecast as f
WHERE f.latitude = round(42.718104*100/25)::integer*25 AND f.longitude = round(11.517727*100/25)::integer*25 and
f.modelts >= '2007-08-01 12:00' and f.modelts <= '2008-08-01 12:00' and date_part('hour', f.modelts) = 12
ORDER BY f.modelts

=== Interpolation ===

The following functions are added so that simple linear interpolation for a specific point in space and time can be done in Postgres.

These are '''ecmwf.forecast_interpolate''' and '''ecmwf.analysis_interpolate''' and they take a latitude, longitude and timestamp as parameter.

They can be used in the same way:

select * from ecmwf.forecast_interpolate( 52.35983, 5.615, timestamp '2008-05-19 13:34:00');

select * from ecmwf.analysis_interpolate( 52.35983, 5.615, timestamp '2008-05-19 13:34:00');

And they can also be used in joins as above. These functions select the 8 surrounding points in latitude, longitude and date_time and interpolate linear in latitude, longitude and date_time successively. The '''interpolate''' functions are much slower then the '''nearest''' functions.

Elevation

2015-12-14T15:34:48Z

Judy: /* Usage */

= ELEVATION =

The database includes a global digital elevation model (DEM) developed from the Shutter Radar Topography Mission(SRTM) data (for more information on the data source, see [[links]]). There is a high resolution (srtm3) and low resolution (srtm30) dataset available in the database. The function elevation.srtm_getvalue extracts the land surface elevation (m) stored in the database for a given set of coordinates (longitude and latitude) which is parsed as text in the function. The elevation.srtm_getvalue function will first try the high-resolution dataset for the specific location and the low-resolution if the first did not give a result.

The function can be used in different ways, shown below.

== Usage ==

select elevation.srtm_getvalue(ST_pointfromtext('POINT(<longitude> <latitude>)',4326))

i.e.

select elevation.srtm_getvalue(ST_pointfromtext('POINT(6.99666 49.504)',4326))

or

select elevation.srtm_getvalue (ST_pointfromtext('POINT(86.93333 27.983333)',4326));

To use only high- or low resolution the following can be used resp.

select elevation.srtm3_getvalue(ST_pointfromtext('POINT(6.99666 49.504)',4326)) -- high res
select elevation.srtm30_getvalue(ST_pointfromtext('POINT(6.99666 49.504)',4326)) -- low res

If a user would like to access the elevation data for a series of locations, than the location field can be used, which is a composite, text field including longitude and latitude of each GPS point, do not run this query for very large datasets without testing first on a subset of data.

SELECT device_info_serial, date_time, latitude, longitude, elevation.srtm_getvalue("location") AS DEM
FROM gps.ee_tracking_speed_limited
WHERE device_info_serial = <device_info_serial>

i.e.

SELECT device_info_serial, date_time, latitude, longitude, elevation.srtm_getvalue("location") AS DEM
FROM gps.ee_tracking_speed_limited
WHERE device_info_serial = 60

On borders between cells the south and/or east cell is selected, also between tiles!

Notes :

The 27 tiles for srtm30 are placed in one table.
The 900 tiles from srtm3 are classified according to the srtm30 classification and distributed to
27 tables.

All tiles have been redistributed into rasters of 60 * 60

== Usage within R ==

If a user would like to access the elevation data for a series of locations, than the location field can be used, which is a composite, text field including longitude and latitude of each GPS point, do not run this query for very large datasets without testing first on a subset of data. This function can also be used from with R but needs a slightly different notation (namely removing quotations around the field location). In the following query elevation.srtm_getvalue(location) AS DEM returns the surface elevation (m) and altitude - elevation.srtm_getvalue(location) AS ALT_AGL returns the GPS altitude above the surface (m) for one specific bird and within a range of dates:

SELECT device_info_serial, date_time, latitude, longitude, elevation.srtm_getvalue(location) AS DEM, altitude - elevation.srtm_getvalue(location) AS ALT_AGL
FROM gps.ee_tracking_speed_limited
WHERE device_info_serial = <device_info_serial> and date_time >= 'yyyy-mm-dd hh:mm:ss' and date_time <= 'yyyy-mm-dd hh:mm:ss'

i.e.
SELECT device_info_serial, date_time, latitude, longitude, elevation.srtm_getvalue(location) AS DEM, altitude - elevation.srtm_getvalue(location) AS ALT_AGL
FROM gps.ee_tracking_speed_limited
WHERE device_info_serial = 6015 and date_time >= '2014-06-27 00:00:00' and date_time <= '2014-06-28 00:00:00'

== Links ==

* [http://srtm.csi.cgiar.org/SRTMdataProcessingMethodology.asp srtm3 download site]
* [http://www.dgadv.com/srtm30/ Images from tiles of srtm30]
* [http://www.weblakes.com/lakesdem.html#SRTM3 Images of coverage of srtm3]

== Credits ==

This postgresql adaptation of the SRTM3 v3 is made by F.P. Sluiter. The original 6000*6000 tiles are retiled to 60*60. New tiles containing only no_data values were removed. Dataset is subdivided in 27 tables that cover the globe.

april 2008, University of Amsterdam

Please put the following reference in publications:

"F.P. Sluiter, University of Amsterdam, april 2008, SRTM3 postgresql adaptation ; based on Jarvis A., H.I. Reuter, A. Nelson, E. Guevara, 2006, Hole-filled seamless SRTM data V3, International Centre for Tropical Agriculture (CIAT), available from http://srtm.csi.cgiar.org."

Original documentation:
PROCESSED SRTM DATA VERSION 3

The data distributed here are in ARC GRID, ARC ASCII and Geotiff format, in decimal degrees and datum WGS84. They are derived from the USGS/NASA SRTM data. CIAT have processed this data to provide seamless continuous topography surfaces. Areas with regions of no data in the original SRTM data have been filled in using interpolation methods. A full technical report on this method is in preparation.

DISTRIBUTION

Users are prohibited from any commercial, non-free resale, or redistribution without explicit written permission from CIAT. Users should acknowledge CIAT as the source used in the creation of any reports, publications, new data sets, derived products, or services resulting from the use of this data set. CIAT also request reprints of any publications and notification of any redistributing efforts.

NO WARRANTY OR LIABILITY

CIAT provides these data without any warranty of any kind whatsoever, either express or implied, including warranties of merchantability and fitness for a particular purpose. CIAT shall not be liable for incidental, consequential, or special damages arising out of the use of any data.

ACKNOWLEDGMENT AND CITATION

We kindly ask any users to cite this data in any published material produced using this data, and if possible link web pages to the CIAT SRTM website (http://srtm.csi.cgiar.org).

Citations should be made as follows:

Jarvis A., H.I. Reuter, A. Nelson, E. Guevara, 2006, Hole-filled seamless SRTM data V3, International Centre for Tropical Agriculture (CIAT), available from http://srtm.csi.cgiar.org.

ECMWF

2015-10-06T12:46:16Z

Judy: /* FORECAST and UVA_FORECAST */

== ECMWF ==

Datamodel for the tables in the ECMWF schema.

Data is created by the ECMWF and retrieved from the KNMI ftp site as GRIB files. The original GRIB files can also be retrieved from the flysafe server from :

[https://flysafe-pps.grid.sara.nl/archive/knmi/ecmwf/ ECMWF Data Archive]

More information about the parameters in these GRIB files can be retrieved from the [http://www.ecmwf.int/services/dissemination/3.1/Products_in_GRIB_code.html ECMWF website].

== INVARIABLE ==

The INVARIABLE table contains data that is invariant over time.

{|border=1
|'''LATITUDE''' || degrees * 100 to make this a integer
|-
|'''LONGITUDE''' || degrees * 100 to make this a integer
|-
|Z || 129 Z Geopotential surface/model orography
|-
|LSM || 172 LSM Land sea mask
|-
|LOCATION || Latitude/longitude as GIS location
|}

The GIS LOCATION is only added to this table. This is because always the same values for latitude and longitude are used and then if the GIS location is required for the ANALYSIS or FORECAST tables it can be retrieved by joining to this table on the LATITUDE, LONGUTUDE fields.

== FORECAST and UVA_FORECAST ==

The FORECAST table contains predicted values in steps of 3 hours till 72 hours ahead. The MODELTS is the timestamp for which this prediction applies. Two times a day new Forecast GRIB files will arrive. The newer FORECAST data that arrives for the same timestamp will overwrite the older data.

The FORECAST (and UVA_FORECAST) table itself is a master table that does not contain any data. The inherited tables
FORECAST<YEAR><MONTH> contain the data, but the master FORECAST or UVA_FORECAST table can be used to SELECT the data.

Data till 2009-09 is owned by the UvA and can be accessed by anybody with ECMWF_READ rights by using the FORECAST_UVA table. The FORECAST table itself also references data after 2009-09 that is owned by RNLAF for which special access rights ECMWF_RNLAF_READ are required.

{|border=1
|'''MODELTS'''||Time for which this forecast applies = model run + step hours
|-
|'''LATITUDE'''||degrees * 100 to make this a integer
|-
|'''LONGITUDE'''||degrees * 100 to make this a integer
|-
|STEP||model step (smaller is more recent forecast)
|-
|SSRD||169 SSRD surface solar radiation downwards accumulated
|-
|TP||228 TP total precipitation accumulated
|-
|SP||134 SP surface pressure
|-
|BLH||159 BLH boundary layer height
|-
|TCC||164 TCC total cloud coverage
|-
|U10||165 10U 10 meters wind U-component m/s
|-
|V10||166 10V 10 meters wind V-component
|-
|T2||167 2T 2 metres air temperature
|-
|D2||168 2D 2 metres dew point temperature
|-
|LCC||186 LCC low cloud coverage
|-
|''Pressure level 1000''||
|-
|T_1000||130 T temperature
|-
|U_1000||131 U U-wind component
|-
|V_1000||132 V V-wind component
|-
|GH_1000||156 GH geopotential height
|-
|R_1000||157 R relative humidity
|-
|''Pressure level 925''||
|-
|T_925 real||130 T temperature
|-
|U_925 real||131 U U-wind component
|-
|V_925 real||132 V V-wind component
|-
|GH_925 real||156 GH geopotential height
|-
|R_925 real||157 R relative humidity
|-
|''Pressure level 850''||
|-
|T_850||130 T temperature
|-
|U_850 real||131 U U-wind component
|-
|V_850 real||132 V V-wind component
|-
|GH_850 real||156 GH geopotential height
|-
|R_850 real||157 R relative humidity
|-
|''Pressure level 700''||
|-
|T_700 real||130 T temperature
|-
|U_700 real||131 U U-wind component
|-
|V_700 real||132 V V-wind component
|-
|GH_700 real||156 GH geopotential height
|-
|R_700 real||157 R relative humidity
|}

== ANALYSIS and UVA_ANALYSIS ==

The ANALYSIS and UVA_ANALYSIS table contains values from the Analysis GRIB files at 18, 00, 06 and 12 UTC

Data till 2009-09 is owned by the UvA and can be accessed by anybody with ECMWF_READ rights by using the ANALYSIS_UVA table. The ANALYSIS table itself also references data after 2009-09 that is owned by RNLAF for which special access rights ECMWF_RNLAF_READ are required.

{|border=1
|'''MODELTS'''||Time for which this forecast applies = model run
|-
|'''LATITUDE'''||degrees * 100 to make this a integer
|-
|'''LONGITUDE'''||degrees * 100 to make this a integer
|-
|SP||134 SP surface pressure
|-
|TCC||164 TCC total cloud coverage
|-
|U10||165 10U 10 meters wind U-component
|-
|V10||166 10V 10 meters wind V-component
|-
|T2||167 2T 2 metres air temperature
|-
|D2||168 2D 2 metres dew point temperature
|-
|LCC||186 LCC low cloud coverage
|-
|''Pressure level 1000''||
|-
|T_1000||130 T temperature
|-
|U_1000||131 U U-wind component
|-
|V_1000||132 V V-wind component
|-
|GH_1000||156 GH geopotential height
|-
|R_1000||157 R relative humidity
|-
|Pressure level 925||
|-
|T_925||130 T temperature
|-
|U_925||131 U U-wind component
|-
|V_925||132 V V-wind component
|-
|GH_925||156 GH geopotential height
|-
|R_925||157 R relative humidity
|-
|Pressure level 850||
|-
|T_850||130 T temperature
|-
|U_850||131 U U-wind component
|-
|V_850||132 V V-wind component
|-
|GH_850||156 GH geopotential height
|-
|R_850||157 R relative humidity
|-
|Pressure level 700||
|-
|T_700||130 T temperature
|-
|U_700||131 U U-wind component
|-
|V_700||132 V V-wind component
|-
|GH_700||156 GH geopotential height
|-
|R_700||157 R relative humidity
|}

== COMBINED ==

If you want to use Analysis data if available for a timestimp but otherwise Forecast, the COMBINED VIEW can be used. If a ANALYSIS value is available for a timestamp and latitude/longitude the ANALYSIS value will be used, otherwise the FORECAST value is given.

== Historical data ==

In addition ongoing process of adding ECMWF data since November 11 2007, a historical data set has been loaded. That historical dataset is stored in additional inherited tables FORECASTHISTORY and
ANALYSISHISTORY which inherit from FORECAST and ANALYSIS respectively.

The FORECASTHISTORY does not always contain TP (228 TP total precipitation accumulated). Instead of this it contains the following values :

{|border=1
|LSP||142 LSP Large scale precipitation
|-
|CP||143 CP Convective precipitation
|}

The historical dat sets span the following periods and locations

{|border=1
|'''Start'''||'''End'''||'''Area'''||'''Grid'''
|-
|2007-01-01||2007-11-15||62degN 10degW 335degN 20degE||0.25x0.25deg
|-
|2003-02-15||2003-06-15||62degN 10degW 335degN 20degE||0.5x0.5deg
|-
|2005-01-01||2006-12-31||62degN 10degW 335degN 20degE||0.5x0.5deg
|}

== Helpful Functions & Queries ==

The following functions have been defined in Postgres to select the nearest gridpoint in spacetime for the ECMWF analysis and forecast tables.

These are '''ecmwf.forecast_nearest''' and '''ecmwf.analysis_nearest''' and they take a latitude, longitude and timestamp as parameter.

They can be used in the following way, which will return all the fields in the relevant ECMWF data:

SELECT * FROM ecmwf.forecast_nearest( 52.35983, 5.615, timestamp '2008-05-19 13:34:00');

SELECT * FROM ecmwf.analysis_nearest( 52.35983, 5.615, timestamp '2008-05-19 13:34:00');

Note that the model latitude and model longitude are returned as integers and multiples of 100. To return latititude and longitude as a decimal (for example to compare to the latitude and longitude of your GPS data) you need to first convert these fields to decimals:

SELECT f.latitude/100::decimal, f.longitude/100::decimal
FROM ecmwf.forecast_nearest( 52.35983, 5.615, timestamp '2008-05-19 13:34:00') f;

These function can for example be used in a join as follows :

SELECT g.ptt_id
, g.date_time
, g.latitude
, g.longitude
, g.speed
, g.course
, g.altitude
, (ecmwf.forecast_nearest( g.latitude, g.longitude,g.date_time)).*
FROM gps.gps_tracking_data g
, gps.individual i
, gps.track_session t
WHERE g.ptt_id = t.ptt_id
AND g.date_time >= t.start_date
AND i.ring_number = t.ring_number
AND g.parser_qc = 1
AND g.ptt_id = 41745
AND g.latitude <> 0
ORDER BY g.date_time;

Although these functions were converted to PLPgSQL to speed them up, a faster ways to retrieve the same data is with (thanks to Floris Sluiters):

SELECT ptt_id, date_time, latitude, longitude, speed, course, altitude, (x::ecmwf.forecast).*
FROM (SELECT
g.*
,((SELECT f
FROM ecmwf.forecast f
WHERE f.latitude = round(g.latitude * 100 / 25)::integer * 25
AND f.longitude = round(g.longitude * 100 / 25)::integer * 25
AND f.modelts = TIMESTAMP 'epoch' + round(EXTRACT(EPOCH FROM g.date_time)
/ (60*60*3))::integer * INTERVAL '3 hours' limit 1)::ecmwf.forecast) as x
FROM gps.gps_tracking_data g
, gps.individual i
, gps.track_session t
WHERE g.ptt_id = t.ptt_id
AND g.date_time >= t.start_date
AND i.ring_number = t.ring_number
AND g.parser_qc = 1
AND g.ptt_id = 41745
AND g.latitude <> 0
ORDER BY g.date_time) t

Below is example query that retrieves data for a model grid point closest to user provided coordinates and for one model time stamp (eg. 12:00) for a series of consecutive days

SELECT f.modelts, f.latitude, f.longitude, f.blh
FROM ecmwf.forecast as f
WHERE f.latitude = round(42.718104*100/25)::integer*25 AND f.longitude = round(11.517727*100/25)::integer*25 and
f.modelts >= '2007-08-01 12:00' and f.modelts <= '2008-08-01 12:00' and date_part('hour', f.modelts) = 12
ORDER BY f.modelts

=== Interpolation ===

The following functions are added so that simple linear interpolation for a specific point in space and time can be done in Postgres.

These are '''ecmwf.forecast_interpolate''' and '''ecmwf.analysis_interpolate''' and they take a latitude, longitude and timestamp as parameter.

They can be used in the same way:

select * from ecmwf.forecast_interpolate( 52.35983, 5.615, timestamp '2008-05-19 13:34:00');

select * from ecmwf.analysis_interpolate( 52.35983, 5.615, timestamp '2008-05-19 13:34:00');

And they can also be used in joins as above. These functions select the 8 surrounding points in latitude, longitude and date_time and interpolate linear in latitude, longitude and date_time successively. The '''interpolate''' functions are much slower then the '''nearest''' functions.

UvA-BiTS Tracking Data

2015-04-09T12:12:36Z

Judy: /* gps.ee_nest_limited */

== Tracking data ==

'''Note:''' These views will show no rows for trackers not yet attached to birds.

=== ''gps.''ee_tracking_speed_limited ===

Shows the tracking data that your projects allow you to see.

Each row represents a GPS fix as it is interpreted from the data that comes via the GPS tracker, plus it adds speeds.

It replaces old views gps.uva_tracking_limited, gps.uva_tracking_speed_3d_limited and gps.uva_tracking_speed. This table structure has ''not'' changed with respect to our previous database.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker that reported the fix. Part of the unique identifier of the GPS fix.
|-
|date_time||timestamp||NO||Time when the fix was taken. UTC (i.e.: no timezone). Part of the unique identifier of the GPS fix.
|-
|latitude||float8||YES||Latitude of the GPS fix, decimal degrees.
|-
|longitude||float8||YES||Longitude of the GPS fix, decimal degrees.
|-
|altitude||int4||YES||Altitude above sea level measured by GPS tag in meters
|-
|pressure||int4||YES||Pressure measured by GPS tag sensor in Pascals
|-
|temperature||float8||YES||Temperature measured by GPS tag sensor in Celsius
|-
|h_accuracy||float8||YES||Horizontal accuracy (meters)
|-
|v_accuracy||float8||YES||vertical accuracy (meters)
|-
|x_speed||float8||YES||meters/second
|-
|y_speed||float8||YES||meters/second
|-
|z_speed||float8||YES||meters/second
|-
|gps_fixtime||float8||YES||GPS fix time (also known as Time to Fix) in seconds
|-
|userflag||int4||YES||Acceptable = 0; Data flagged as unacceptable by user if not equal to 0.
|-
|satellites_used||int2||YES||Number of satellites used for fix the GPS measurement
|-
|positiondop||float8||YES||Positional DOP (Dilution of Precision), unitless
|-
|speed_accuracy||float8||YES||
|-
|location||geometry||YES||postgreSQL geometry; to allow GIS operations
|-
|vnorth||float8||YES||- vx * sin(lat) * cos(long) - vy * sin(lat) * sin (long) + vz * cos(lat), m/s
|-
|veast||float8||YES||- vx * sin(lon) + vy * cos(lon), m/s
|-
|vdown||float8||YES||- vx * cos(lat) * cos(lon) - vy * cos(lat) * sin(lon) - vz * sin(lat), m/s
|-
|speed||float8||YES||- vx * sin(lon) + vy * cos(lon)
|-
|speed_3d||float8||YES|| sqrt(d.x_speed^2 + d.y_speed^2 + d.z_speed^2), m/s
|-
|speed_2d||float8||YES|| Speed on 2D, m/s
|-
|direction||numeric||YES|| Heading in degrees; 0 is north, 90 is east
|-
|altitude_agl||float8||YES|| Reported GPS altitude MINUS terrain elevation on the coordinate based on Digital Elevation Model data used in the e-ecology database (for further information see: [[Elevation]]
|}

=== ''gps.''ee_acceleration_limited ===

Acceleration is measured in blocks of 1-200 measurements (records). The date_time mentioned in this block is the date and time of the first acceleration measurement. Sequential measurements within a block are numbered with an index 1... (max 200). Every row represents one measurement in 3 directions (X,Y,Z). The order is important within each block, so you should always sort the acceleration measurements by block and index. If you need the exact time for each row, you can calculate it from the index, the sample frequency (which is almost always 20 Hz) and the date_time of the first row in the block.

The date_time of the first block is the time of the GPS measurement preceding the accelerometer measurement. If there is no valid GPS measurement, this time is calculated from the last GPS measurement and the time of the internal clock of the tracker.

Therefore, to know all valid dates and times that mark a valid block for a tracker, you need to consider all of the values in column date_time in gps.ee_tracking_speed_limited for the given tracker, column date_time on gps.ee_acc_start_limited, and all the values on column date_time for that tracker.

This table structure has ''not'' changed with respect to our previous database.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker that reported the fix. Part of the unique identifier of the GPS fix.
|-
|date_time||timestamp||NO||Time of the acceleration block that this measurement belongs to (time in UTC, no time zone).
|-
|index||int2||NO||Sorting order of the measurement within the acceleration block (each block starts at 0).
|-
|x_acceleration||int2||YES||Acceleration measured on the x (or surge) axis (range between -2666 and 2666).
|-
|y_acceleration||int2||YES||Acceleration measured on the y (or sway) axis (range between -2666 and 2666).
|-
|z_acceleration||int2||YES||Acceleration measured on the z (or heave) axis (range between -2666 and 2666).
|}

=== ''gps.''ee_acc_start_limited ===

Each row represents the start of a block of acceleration measurements when a GPS fix is not available at that acceleration block's time.

If there is no valid GPS measurement, this time is calculated from the last GPS measurement and the time of the internal clock of the tracker.

To know all valid dates and times that mark a valid block for a tracker, you need to consider all of the values in column date_time on gps.ee_tracking_speed_limited for the given tracker, column date_time on gps.ee_acc_start_limited, and all the values on column date_time for that tracker. And the order is important within each block, so you should sort the acceleration measurements by block and index.

Added in firmware 1.0.2.7.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker that reported the acceleration measurement.
|-
|date_time||timestamp||NO||Time to mark the start of the acceleration block.
|-
|line_counter||int4||NO||Number of lines associated with this set of accelerations.
|-
|timesynced||int2||YES||The source of the time that marks the acceleration block (0=not synced, 1=user set, 2=Gps Synced).
|-
|ascii||int4||YES||Acceleration measurement interval.
|-
|accsn||int4||YES||Number of samples (0-65536).
|-
|f||int2||YES||Acceleration sampling frequency (0=20Hz, 1=10Hz, 2=5Hz, 3=1Hz).
|}

== Bird data ==
=== ''gps.''ee_individual_limited ===

Each row shows bird's details for all birds that your projects allow you to see.

This view replaces the old gps.uva_individual_limited.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|ring_number||varchar||NO||Ring number.
|-
|species_latin_name||varchar||NO||Latin name of the species of the bird.
|-
|colour_ring||varchar||YES||String for ring colour.
|-
|sex||varchar||NO||Sex (M=male, F=female, X=unknown/unspecified)
|-
|mass||numeric(5,0)||YES||Mass (grams)
|-
|start_date||timestamp||NO||Date when the bird was first tracked with an UvA-BiTS logger.
|-
|end_date||timestamp||NO||Date at which the bird is no longer relevant for the tracking project.
|-
|remarks||text||YES||Text for user remarks
|-
|individual_id||int8||NO||Bird unique id. Included just to enable joins.
|}

=== ''gps.''ee_species_limited ===

Each row represents a species that you can assign to your birds to include them in your project. No bird in the system can exist without being linked to one of the species from this list. If you are tagging a bird whose species does not exist in the list, please [http://www.uva-bits.nl/contact/ contact] us.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|abbr||varchar||NO||Unique abbreviation for this species.
|-
|english_name||varchar||YES||Name of the species in English.
|-
|latin_name||varchar||NO||Name of the species in Latin. It is always filled in.
|-
|species_id||int8||NO||Unique id. Added to allow joins.
|}

=== ''gps.''ee_nest_limited ===
This view shows nests you have access to. A nest is always for a bird with a tracker (so a track session must exist for that bird), and must contain a short name (a reference name for you to use) and a latitude and longitude pair.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|reference_name||varchar||NO||A string with a useful reference for you. We recommend its being unique for each nest in a project, but the system does NOT enforce it.
|-
|latitude||numeric||YES||Latitude of the coordinates where the nest was found, decimal degrees.
|-
|longitude||numeric||YES||Longitude of the coordinates where the nest was found, decimal degrees.
|-
|start_date_time||timestamp||NO||When the nest was spotted.
|-
|end_date_time||timestamp||NO||When the nest ceases to be relevant.
|-
|found_by_whom||varchar||YES||Place to credit people involved in finding the nest.
|-
|remarks||text||YES||Free text.
|-
|location||bytearray||YES||Calculated Geometry of the coordinates where the nest was found.
|-
|nest_id||int8||NO||Unique id. Added to allow joins to nest_inhabitant_limited.
|}

=== ''gps.''ee_nest_inhabitant_limited ===
This view shows who lives on which nest, for birds you have access to.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|reference_name||varchar||NO||A string belonging to the nest that you can use to identify that nest. Because it is NOT enforced to be unique, DO NOT USE THIS FIELD FOR JOINING.
|-
|ring_number||varchar||NO||The ring number of a bird on a nest.
|-
|device_info_serial||int8||NO||Following the link from a bird to her track sessions, and considering the nest's start and end time, the bird's trackers within that period are resolved so that they can be shown here. This is the device_info_serial of a tracker.
|-
|key_name||varchar||NO||Following the link from a bird to her track sessions, and considering the nest's start and end time, the track sessions' projects within that period are resolved so that they can be shown here. This is the key_name of a project.
|-
|nest_id||int8||NO||Reference to the nest id. Added to allow joins.
|-
|individual_id||int8||NO||Reference to the individual id. Added to allow joins.
|-
|track_session_id||int8||NO||Reference to a resolved track session id. Added to allow joins.
|}
== Tracker and project data ==

=== ''gps.''ee_project_limited ===

Each row shows the details of each project, for the projects you are allowed to see.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||Unique key name, with the form <species>_<place>
|-
|station_name||varchar||NO||Where the receiving station is
|-
|start_date||timestamp||NO||Project start date
|-
|end_date||timestamp||NO||Date at which the project is not relevant any more
|-
|description||text||YES||String for free description. Updatable by the user.
|-
|project_id||int8||NO||Project unique id. Included just to enable joins.
|-
|parent_id||int8||YES||Reference to the parent project in the hierarchy.
|}

=== ''gps.''ee_tracker_limited ===

Each row shows tracker details for all trackers that your projects allow you to see.

This view replaces the old gps.uva_device_limited.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int8||NO||Serial number of the tracker
|-
|firmware_version||varchar||YES||String for firmware version
|-
|mass||numeric(4,2)||YES||Mass (in grams)
|-
|start_date||timestamp||NO||Date before which the device is not relevant
|-
|end_date||timestamp||NO||Date at which the device is not relevant any more
|-
|x_o||numeric(30,6)||YES||Accelerometer calibration: offset on the x (or surge) axis
|-
|x_s||numeric(30,6)||YES||Accelerometer calibration: sensitivity on the x (or surge) axis
|-
|y_o||numeric(30,6)||YES||Accelerometer calibration: offset on the y (or sway) axis
|-
|y_s||numeric(30,6)||YES||Accelerometer calibration: sensitivity on the y (or sway) axis
|-
|z_o||numeric(30,6)||YES||Accelerometer calibration: offset on the z (or heave) axis
|-
|z_s||numeric(30,6)||YES||Accelerometer calibration: sensitivity on the y (or heave) axis
|-
|tracker_id||int8||NO||Tracker unique id. Included just to enable joins.
|}

=== ''gps.''ee_track_session_limited ===

A track session links a tracker and a bird for a given period of time. Therefore, it is '''mandatory for projects''' to supply this piece of '''metadata''' before they can see any tracking data related to a tracker they own.

In order to create a track session, the project must create first an entry in the system for the bird that they are attaching the tracker to, and this is the way for the UvA-BiTS community to make sense of the tracking data that we are receiving. This is why providing the metadata is so important.

By editing the start and end dates, administrators of the project that this track session belongs to, can delimit (and therefore, establish) when the coupling makes sense (or is to be understood as useful).

Whenever a tracker is attached to a bird, a track session must be started (start_date) to indicate that the tracking data that is received is already live data. If the tracker is separated from the bird, the tracking data ceases to make sense, so the track session's end date should be set to denote it has ended. New track sessions can be started if the tracker is re-attached to a bird (possibly the same as before, or maybe a different one) afterwards.

Each row shows track session's information. There exists a row for each track session that your projects allow you to see.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||The project that the track session belongs to.
|-
|device_info_serial||int8||NO||Serial number of the tracker this track session involves.
|-
|ring_number||varchar||NO||Ring number of the bird that this track session involves.
|-
|start_date||timestamp||NO||Date before which the track session is not relevant
|-
|end_date||timestamp||NO||Date at which the track session is not relevant any more
|-
|remarks||text||YES||Free text for comments
|-
|start_latitude||numeric(11,8)||NO||Latitude where the bird was tagged, decimal degrees
|-
|start_longitude||numeric(11,8)||NO||Longitude where the bird was tagged, decimal degrees
|-
|project_id||int8||NO||Reference to the project this track session belongs to. Included just to enable joins.
|-
|tracker_id||int8||NO||Reference to the tracker this track session involves. Included just to enable joins.
|-
|individual_id||int8||NO||Reference to the individual this track session involves. Included just to enable joins.
|-
|track_session_id||int8||NO||Unique identifier of the track session. Included just to enable joins.
|}

== Relation to projects ==

=== ''gps.''ee_user_limited ===

Each row shows a colleague in a project that you belong to, indicating her role and status there.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||Project name
|-
|email||varchar||NO||e-mail address
|-
|login_name||varchar||NO||Login name
|-
|name||varchar||NO||(first) name
|-
|surname||varchar||NO||Surname
|-
|role||varchar||NO||Role in the project
|-
|status||varchar||NO||Status of the membership
|-
|user_id||int8||NO||Unique identifier of the user. Included to enable joins.
|}

=== ''gps.''ee_tracker_ownership_limited ===

Each row that you see expresses that a tracker belongs to one of the projects you are involved with.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||The project that the tracker belongs to.
|-
|device_info_serial||int8||NO||Serial number of the tracker.
|-
|project_id||int8||NO||Reference to the project. Included just to enable joins.
|-
|tracker_id||int8||NO||Reference to the tracker. Included just to enable joins.
|}

=== ''gps.''ee_individual_ownership_limited ===

Each row that you see expresses that a bird belongs to one of the projects you are involved with.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||The project that the individual belongs to.
|-
|ring_number||varchar||NO||Ring number of the bird.
|-
|project_id||int8||NO||Reference to the project. Included just to enable joins.
|-
|individual_id||int8||NO||Reference to the individual. Included just to enable joins.
|}

== Engineering data ==

'''Note:''' These views will show no rows for trackers not yet attached to birds.

=== ''gps.''ee_comm_limited ===

Each row indicates status and result of communications with ground stations for each tracker that your project allows you to see.

Only for newer firmware (v.1.0.1.4 and higher).

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
| device_info_serial || int4 || NO || Serial number of the tracker.
|-
| date_time || timestamp || NO || timestamp of start communication
|-
| date_time_end || timestamp ||YES|| timestamp of end communication
|-
| date_time_utc || timestamp ||YES|| date_time in utc (if available)
|-
| date_time_end_utc || timestamp ||YES|| date_time_end in utc (if available)
|-
| action || text||YES|| JOINED / RESCHEDULE ??
|-
| bytes_communicated || int4 ||YES|| Number of bytes communicated (total of all bytes per DATA line)
|-
|status || text ||YES|| READY, TIMEOUT, LOWPOWER, MISSING
|-
| network_join_time || int4||YES||milliseconds
|-
| number_identical_lines || int4 ||YES|| Number identical lines in data block. Gives indication
|-
| non_ascii_char || int4 ||YES|| Number of non ascii characters (indicated by question-mark ?)
|-
| sts_time || timestamp ||YES|| derived from S0 hh,mi
|-
| timesynced || int2||YES|| TimeSynced (0=not sy:wnced, 1=user set, 2=Gps Synced)
|-
|devicemode || int2||YES||DeviceMode (0-99). This describes whether the device is in a special operational mode. 0 = normal operational mode; 80 hex = Memory save mode (memory is almost full and all activity is drastically reduced so the circular memory probably won't be overwritten or, if it does, it will be kept to a minimum)
|-
| vbat || float8 ||YES|| Battery voltage in V (0-~4.200V)
|-
|logb || int4 ||YES|| logged bytes, waiting for download (0-4Mbyte)
|-
| logp || int2 ||YES|| Logged percentage (units 0.1%) of memory waiting to be downloaded (0-1000)
|-
| gpsii || int4 ||YES|| GpsIval, current (0-65535)
|-
|comii || int4||YES|| ComIval, current (0-65535)
|-
|accii || int4||YES|| AcceleroIval, current (0-65535)
|-
|swddd || int4||YES|| User switches, current (0-65535)
|-
|last_gps_fix ||timestamp||YES||timestamp of last fix
|-
|latitude||float8||YES||Latitude of last fix. Decimal degrees
|-
|longitude||float8||YES||Longitude of last fix. Decimal degrees
|}

=== ''gps.''ee_energy_limited ===

Only for newer firmware.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker
|-
|date_time||timestamp||NO||Date and time of the reported data
|-
|timesynced ||int2||YES||TimeSynced (0=not synced, 1=user set, 2=Gps synced)
|-
|devicemode ||int2||YES||(0-99)
|-
|vsll ||float8||YES||Solar Voltage during charging in mV (0-~2500)
|-
|vbat ||float8||YES||Battery Voltage in V (0-~4.200)
|-
|ssw ||int2||YES|| Solar Cell Switches ... (0-255) fw1022 and up
|-
| ||||YES||extended fields if special user switch is on
|-
|vsoo ||float8||YES|| Solar voltage open in V (0-~2.500)
|-
|vdrf ||float8||YES||reference voltage Dac in V (0-~2.435)
|-
|vchr ||float8||YES||charge current voltage
|-
|g ||int2||YES|| Gps battery threshold OK
|-
|c || int2||YES||Com battery threshold OK
|-
|m || int2||YES||Charge Mode (0,1,2 Com battery threshold OK
|-
|pressure || int4||YES|| in pascal (1mBar = 100Pa, 1pressuremeter = ~12Pa)
|-
|temperature || float8||YES|| in celcius degrees
|}

=== ''gps.''ee_settings_limited ===

Settings are either from C0-C1-C9 lines or parsed from uplink communication lines.
If C0-C1 is parsed its values are compared with the previous value and if the settings are the same, then no new records need to be added.
If uplink settings are parsed, then wait until the last setting has been seen (DL command or EOF); then store all the new settings in one record. Only for newer firmware.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||
|-
|date_time||timestamp||NO||Derived timestamp from Hh:mi fields in C0 combined with computer timestamp in JOINED line
|-
|timesynced || int2 ||YES||
|-
|timezone_time || int2 ||YES||
|-
|swddd || int4||YES|| SpecialUserSwitches
|-
|gpsnavmode || int2||YES|| GpsNavMode 0-8. This is the Kalman filter used for GPS navigation mode often called the "Dynamic platform model". The following codes are provided by the GPS chip manufacturer for the UBlox GPS chip LEAS4S: 1 = stationary, 2 = pedestrian, 3 = automotive, 4 = sea, 5 = airborne 1G, 6 = airborne 2G, 7 = airborne 3G. The UBlox GPS chip LEAS6S uses the following codes: 0 = portable, 2 = stationary, 3 = pedestrian, 4 = automotive, 5 = sea, 6 = airborne with < 1g acceleration, 7 = airborne with < 2g acceleration, 8 = airborne with < 4g acceleration
|-
|gpsreceivermode || int2 ||YES|| GpsReceiverNode 0-3
|-
|gpsfixtime || int2||YES|| GpsFixTime 0-255
|-
|gps_ivaln_on_h || int2||YES|| hours Gps IvalN on
|-
|gps_ivaln_on_m || int2||YES|| minutes Gps IvalN on
|-
|gps_ivaln || int4||YES|| GpsIvalN (0-65530)
|-
|gps_ivale_on_h || int2||YES|| hours Gps IvalE on
|-
|gps_ivale_on_m || int2||YES|| minutes GpS IvalE on
|-
|gps_ivale || int4||YES|| GpsIvalE
|-
|com_ivaln_on_h || int2||YES|| hours Com IvalN on
|-
|com_ivaln_on_m || int2||YES|| minutes Com IvalN on
|-
|com_ivaln || int4||YES|| ComIvalN (0-65530)
|-
|com_ivale_on_h || int2||YES|| hours Com IvalE on
|-
|com_ivale_on_m || int2||YES|| minutes Com IvalE on
|-
|com_ivale || int4||YES|| ComIvalE
|-
|accii || int4||YES|| Accelero measurement interval
|-
|accbn || int4||YES|| Accelero number of blocks
|-
|accsn || int4||YES|| Accelero number of samples (0-65536)
|-
|f || int2||YES||Accelero sample frequency (0=20Hz, 1=10Hz, 2=5Hz, 3=1Hz)
|-
|pdp || int2||YES|| PdopMsk (3-250)
|-
|pac||int2||YES|| PaccMsk (10-100)
|-
|tdp ||int2||YES|| TdopMsk (3-250)
|-
|tac ||int2||YES|| TaccMsk (30-300)
|-
|ga || int2||YES|| Additional Fixes GpsIval mode (3-25) before interval fix is logged
|-
|aol || int2||YES||AccMemOffLev (100 - 950) units 0.1%; if Memfull above this level all Accelero activity will be overruled and switched off untill MemFull < level - 5
|-
|eventnum || text||YES|| 12 eventnumbers (0-0xff) only printed if not zero, therfore converted to text
|}

=== ''gps.''ee_trackingfile_limited ===

Each row shows the status of the last attempt to parse (process) the raw data of a file for a given tracker, for each tracker that your projects allow you to see.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|id||int8||NO||Unique identifier of the row
|-
|date_reported||date||NO||Date that the file arrived
|-
|filename||varchar||NO||Name of the file
|-
|last_modified_date||timestamp||NO||Date that the file was last modified on
|-
|device_info_serial||int4||YES||Serial number of the tracker this file comes from
|-
|filesize||int8||NO||Size of the file in KB
|-
|parsed_date||timestamp||YES||Date and time that the file was last attempted to be parsed on
|-
|version||int4||NO||Optimistic locking field. Ignore it as a user.
|-
|last_found_date||timestamp||YES||Last time that a process that runs to look for new files saw the file
|-
|n_lines||int4||YES||Count of lines found in the file last time it was processed
|-
|n_data_lines||int4||YES||Count of actual data lines, out of the total lines
|-
|n_tracking_records||int4||YES||Count of GPS fixes found during the parsing process
|-
|n_communication_records||int4||YES||Count of communication records found during the parsing process
|-
|n_energy_records||int4||YES||Count of energy records found during the parsing process
|-
|n_acceleration_records||int4||YES||Count of acceleration records found during the parsing process
|-
|n_acceleration_start_records||int4||YES||Count of acceleration start records found during the parsing process
|-
|n_errors||int4||YES||Count of errors encountered during the parsing process
|-
|n_settings_records||int4||YES||Count of settings records found during the parsing process
|-
|filepath||text||NO||Directory path of the file on the server
|}

=== ''gps.''ee_warning_limited ===

Since firmware version 1.0.2.7 this table was added for events that do not occur very often (C10, C20) or errors that occur during parsing.

For example all CRC failures can be selected with :

select * from gps.ee_warning_limited
where warning = 'CRC8 failure'
order by device_info_serial;

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker
|-
|filename||text||NO||File name
|-
|linenumber||integer||YES||line number in file
|-
|warning||text||YES||The warning message
|-
|line||text||YES||The actual line contents in the file
|}

== Sharings ==

Projects can now share trackers with other projects during periods of time. This means that people in the receiving projects can access information that these shared trackers provide during those time periods.

The following views allow you to see this shared information:

* Tracking data
** ''gps.''ee_shared_tracking_speed_limited: tracking information (gps, speed), during the shared period, for trackers shared
** ''gps.''ee_shared_acc_start_limited: acceleration start information, during the shared period, for trackers shared
** ''gps.''ee_shared_acceleration_limited: acceleration information for trackers shared
* ''Meta'' data
** ''gps.''ee_shared_tracker_limited: information about each shared tracker
** ''gps.''ee_shared_individual_limited: information about birds with shared trackers
** ''gps.''ee_shared_track_session_limited: information about track sessions which involve shared trackers
* Relation to projects
** ''gps.''ee_shared_tracker_sharing_limited: view information for each sharing that your projects allow you to see

== Accelerometer calibration ==

The accelerometer measures the acceleration in 3D (20 times per second). Acceleration is caused by gravity (static component) and by a change in speed (dynamic component).
When the logger is static in horizontal position (solar cells up), gravity (green arrow) shows a heave (vertical acceleration, z) of +1 g (1 g = 9.81 m.s-2) while surge and heave are both 0. When the front of the logger is vertically pointing down (forward acceleration, x), surge is +1g and when it is rolled to the right (sidewards acceleration, y) sway is +1g. Due to gravity, the pitch of a logger is thus translated into the values of heave, surge and sway (or z, x, y).

However, the values are measured in mV instead of g. Before delivering the GPS-loggers, they have been calibrated at UvA. The acceleration (A) expressed in g is calculated from the measured value (M) and the calibration factors offset (''O'') and sensitivity (''S'').

Thus acceleration for heave (z), surge (x) and sway (y) is calculated as:
Az = (Mz-Oz)/Sz ; Ax = (Mx-Ox)/Sx; Ay = (My-Oy)/Sy ;

The values of ''O'' (in general ranging from 0-60, sometimes even 300) and ''S'' (ranging from 1300-1450) are available in the device table of the data base. If not available, use the factory defaults ''O'' = 0 and ''S'' = 1365.

{|
| [[File:accelero1.png|300px]]||[[File:accelero2.png|300px]]|
|}

In the figure above heave (z) is shown in green, surge(x) in red and sway (y) in blue, all values are expressed in g (on the y-axis).

== Data processing and filtering ==
The raw log files are processed before being entered into the database. Each firmware version has a dedicated parser for the task. In certain cases data that is found in the log files is not stored in the database. This can occur for two reasons:

1. The data did not pass the CRC8 checksum (cyclic redundancy check). This is a standard procedure designed to detect accidental changes to raw computer data.

2. If the data has an invalid signature. For example, if a certain data format (eg numeric) or number of values (eg 4 digits) is expected but not returned, this record is not stored in the database.

When a line is deleted, the log file name, line number and the type of error is stored in the table gps.uva_warning102

== Information about GPS receivers ==
The first series of tags used the UBlox LEAS4S chip (~up to tag 541), newer tags use UBlox LEAS6S. The series developed in 2012 will use the Max6G receivers. From 2012 receiver versions will be stored in the database.
The UBX protocol is used for programming firmware, packaging and processing data.
For more information about the receivers themselves see the ublox documentation.
Ublox 4 Protocol specification [http://www.u-blox.com/images/downloads/Product_Docs/ANTARIS_Protocol_Specification%28GPS.G3-X-03002%29.zip]
Ublox 6 Protocol specification [http://www.u-blox.com/images/downloads/Product_Docs/u-blox6_ReceiverDescriptionProtocolSpec_%28GPS.G6-SW-10018%29.pdf]

Useful Queries

2015-03-13T07:59:13Z

Judy: /* Calculating summary statistics */

= UvAGPS functions =

== get_uvagps_track_speed ==

Compute distance (meters), time-interval (time stamp hh:mm:ss), speed (meters/second) and heading (degrees) for a series of subsequent tracking entries for a specific device_info_serial.
For the first entry in the sequence these values will always be missing because they are computed as the difference between subsequent rows.

Usage examples :

<pre>
select * from gps.get_uvagps_track_speed(<device_info_serial>);

select * from gps.get_uvagps_track_speed(119, '2008-01-01 00:00:00', '2010-01-01 00:00:00');

select t.*, a.distance, a.interval, a.speed, a.direction
from gps.get_uvagps_track_speed(119, '2008-01-01 00:00:00', '2010-01-01 00:00:00') a
join gps.ee_tracking_speed_limited t using(device_info_serial, date_time);
</pre>

Distances and heading are calculated with spherical correction with the Haversine formula as described [http://http://www.movable-type.co.uk/scripts/latlong.html here].
The heading is the initial heading. PostGIS does not calculate the heading, so therefore we don't use PostGIS.

In these functions the records that have been flagged as invalid by setting the '''userflag''' to 1 are not taken into account and discarded.
If speed calculations are required for all records an additional boolean argument '''get raw data''' can be given. i.e.

<pre>
select * from gps.get_uvagps_track_speed(119, '2008-01-01 00:00:00', '2010-01-01 00:00:00', true);

select * from gps.get_uvagps_track_speed(119, true);
</pre>

== get_uvagps_track_distance ==

Compute distance (meters) for UvAGPS tracks to arbitrary point in lat/long or geometry location

Example usage :

<pre>
select * from gps.get_uvagps_track_distance(119, 52.1897017, 6.1996606);

select * from gps.get_uvagps_track_distance(119, (select location from gps.ee_tracking_speed_limited where device_info_serial = 119 and date_time = '2009-05-07 19:55:08'));

</pre>

Here the distance is calculated with the PostGIS distance_sphere function which uses spherical correction.

== get_uvagps_track_distance_direction ==

Compute distance (meters) and direction (degrees) for UvAGPS tracks from arbitrary point in lat/long or geometry location

Example usage:

<pre>
select * from gps.get_uvagps_track_distance_direction(119, 52.1897017, 6.1996606);
</pre>

Distances and direction are calculated with spherical correction with the Haversine formula as described [http://http://www.movable-type.co.uk/scripts/latlong.html here]. The direction is the direction from the given point.

In this function the records where the '''userflag''' has been set are discarded. To see these records a additional boolean argument for '''show raw data''' should be added like :

<pre>
select * from gps.get_uvagps_track_distance_direction(119, 52.1897017, 6.1996606, true);
</pre>
= Filtering tracking data =

== Select tracking data from the beginning of a track session ==

Select only those points that were collected during a valid track session (i.e. data that begins from the start_date of a track session) including 2D GPS speed. Filter our points where no GPS fix is returned (latitude is not null) and where a user has flagged the data as inappropriate (user_flag <> 1):

SELECT t.device_info_serial, t.date_time, t.latitude, t.longitude, t.altitude, t.temperature, t.speed, t.userflag
FROM gps.ee_tracking_speed_limited t, gps.ee_track_session_limited s
WHERE t.device_info_serial = s.device_info_serial AND t.date_time >= s.start_date AND t.device_info_serial = '''<ID>'''
AND t.latitude IS NOT NULL AND t.userflag <> 1

== Select tracking data and distinguish between distinct GPS deployments ==
In this query you will select the tracking data related to each deployment of a GPS logger or each time an individual bird was tracked. This query is slightly more complex than the query above. If you have used the same logger on more than one individual bird, or have tracked the same individual with more than one logger, this query will enable you to make a distinction between these tracking events by showing both GPS logger id (device_info_serial) and a unique identifier for your individual (eg ring_number or colour_ring). Furthermore, as above, this will only select tracking data from the start and end of a particular track session.

SELECT t.device_info_serial, t.date_time, t.latitude, t.longitude, t.altitude, t.speed_2d, i.colour_ring, s.track_session_id, s.key_name
FROM gps.ee_track_session_limited s
JOIN gps.ee_individual_limited i
ON s.individual_id = i.individual_id
JOIN gps.ee_tracking_speed_limited t
ON t.device_info_serial = s.device_info_serial
AND t.date_time BETWEEN s.start_date AND s.end_date
WHERE s.key_name = ''''<Project name>'''' AND t.userflag = 0
ORDER BY t.device_info_serial, t.date_time

HINT: WHERE s.key_name = 'HG_TEXEL'

== Re-sample tracking data to select data with a predefined interval between fixes ==

The following query provides an example of how to select data points with a predefined interval between each point and not lower. In the following example, a 5 minute interval is selected. In this example all measurements that are taken between each 5 minute interval are left out of the resampled data.

Commented query with examples. Statements followed by -- are comments and are not parsed by SQL.

SELECT *
FROM gps.ee_tracking_speed_limited
JOIN
(
SELECT
device_info_serial,
-- return oldest date_time for each interval
MIN(date_time) date_time,
-- each 5 minutes has a unique identifier
FLOOR(EXTRACT(EPOCH FROM date_time)/(5*60)) intervalid
FROM gps.ee_tracking_speed_limited
GROUP BY device_info_serial, intervalid
) b
USING (device_info_serial, date_time)
WHERE device_info_serial = 355 AND date_time BETWEEN '2010-06-27' AND '2010-06-29'
ORDER BY date_time

Uncommented query, date_time and device_info_serial need to be filled in. Interval is 5 minutes and can be changed.

SELECT *
FROM gps.ee_tracking_speed_limited
JOIN
(
SELECT
device_info_serial,
MIN(date_time) date_time,
FLOOR(EXTRACT(EPOCH FROM date_time)/(5*60)) intervalid
FROM gps.ee_tracking_speed_limited
GROUP BY device_info_serial, intervalid
) b
USING (device_info_serial, date_time)
WHERE device_info_serial = '''<ID>''' AND date_time BETWEEN ''''yyyy-mm-dd'''' AND ''''yyyy-mm-dd''''
ORDER BY date_time

= Calculating summary statistics =
== Total distance ==
A query to determine the total distance a bird has flown is :

select sum(distance)/1000 as total_distance
from gps.get_uvagps_track_speed('''<ID>''');

or for a fixed time period :

select sum(distance)/1000 as total_distance
from gps.get_uvagps_track_speed('''<ID>''', '2010-01-01 00:00:00', '2011-01-01 00:00:00');

== Total number of measurements per tag ==

Count the total number of measurements for each device using the GROUP BY clause:

select device_info_serial, count(*)
from gps.ee_tracking_speed_limited
group by device_info_serial
ORDER BY device_info_serial ASC;

If you want to count the number of measurements with valid GPS fixes you can add a where clause :

select device_info_serial, count(*)
from gps.ee_tracking_speed_limited
where latitude is not null
group by device_info_serial
ORDER BY device_info_serial ASC;

Count the number of GPS fixes within a track session and for all track sessions within a specific project.
In the example below the project name needs to be filled in, for example s.key_name = 'HG_IVES'

WITH
V AS (
SELECT s.device_info_serial, count(t.*) AS point_count, s.track_session_id, s.individual_id
FROM gps.ee_track_session_limited s
JOIN gps.ee_tracking_speed_limited t
ON t.latitude IS NOT NULL
AND s.device_info_serial = t.device_info_serial
AND t.date_time BETWEEN s.start_date AND s.end_date
WHERE s.key_name = ''''< project name >''''
GROUP BY s.track_session_id, s.device_info_serial, s.individual_id
)
SELECT i.colour_ring, V.device_info_serial, V.point_count, V.track_session_id
FROM V
JOIN gps.ee_individual_limited i
ON V.individual_id = i.individual_id
= Extra Metadata =
== Extract all track sessions for a project ==
This query provides all the track sessions for a specific project as well as the selected ID of the individual birds that were tracked. In the example below the project name needs to be filled in, for example s.key_name = 'HG_IVES'

SELECT s.device_info_serial, i.colour_ring, s.track_session_id, s.key_name
FROM gps.ee_individual_limited i
JOIN gps.ee_track_session_limited s
ON s.individual_id = i.individual_id
WHERE s.key_name = ''''< project name >''''
ORDER BY i.colour_ring

E-Ecology Documentation and Guidance

2015-03-02T14:08:06Z

Judy: /* UvA-BiTS Services */

The e-Ecology Virtual Laboratories represent a collaborative effort between the University of Amsterdam Institute for Biodiversity and Ecosystem Dynamics and SARA Computing to manage bird tracking data and develop tools for data exploration, analysis and visualization. This wiki is designed to facilitate access to the data and tools within the Virtual Laboratories and to provide a forum for collaboration.

== UvA-BiTS data ==

:Bird data collected with the UvA-BiTS Bird Tracking System

Note: We are continually working on improving UvA-BiTS services, including data processing and Virtual Lab functionality. In some cases, this has enabled us to extract more information from old GPS log files which is then incorporated into the database. These changes may result in slight differences when you reprocess historic data.

::* [[ DB_Views_2015 | e-Ecology database for general use (views) ]]
::* [[ DB_Tables_2015 | e-Ecology tables behind the views (not accessible for normal use) ]]
::* [[ UvAGPS|UvA-BiTS Data Model]] - OLD
::* [[Connecting to the UvA-BiTS database]]
::* [[Useful Queries]]
::* [[Queries for linking bird tracks to wind and flow data]]

==UvA-BiTS Services==

* Visualization Tools
:: [[BirdView]]: 
:::Access meterological and radar data and overlay with GPS bird tracking data.
:: [[BirdSim]]: 
:::More dynamic version of BirdView that allows the user to scroll through weather data in time and space.
:: [[KMZ Generator|UvA-BiTS KMZ Generator]]: 
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth.
:: [[Tracker Calendar]]: 
:::Calendar overview with daily statistics of GPS-tracker
:: [[Flight Generator]]: 
:::This tool produces GPX files from UvA-BiTS tracking data for dynamic [http://www.doarama.com Doarama] visualizations.

* [[Dashboard]]: These web pages allow UvA-BiTS users to get an overview of the raw data files of GPS bird tracking devices they have access to and to access the dashboard for administering projects.

* [[ Annotation tables ]] for uploading GPS annotation data into your own schema. This service is currently not available.

== Changes in the Database in 2014 ==
'''Important'''
In the summer of 2014 we have '''re-arranged the administrative''' part of UvA-BiTS (and e-Ecology) in the database. End users may have to adapt their programs the following way:
:* The '''uva_*_limited''' views that used to exist in the gps schema are gone. Most of them should have been replaced with an equivalent '''ee_*_limited counterpart'''.
:* The new ee_*_limited views show a superset of the same information as before, but enhanced with new project information.
:* Some '''column names''' in the ee_*_limited views may differ from the previous views.
:* There are new limited views:
:** gps.ee_'''project'''_limited: shows your projects' information
:** gps.ee_'''shared'''_*_limited: set of data you are able to see related to trackers that a project is sharing with a project you belong to
:* Special attention: gps.uva_device_limited is now replaced by '''gps.ee_tracker_limited'''. For each tracker, you now see which project provides you access to that tracker.
:* Also, '''gps.uva_tracking_limited''' is gone. Instead of that we have gps.ee_tracker_speed_limited, which adds speed columns to the original data. This amounts to a merger of gps.ee_tracking_limited, gps.ee_tracking_speed_3d_limited and gps.ee_tracking_speed_limited.

==Other e-Ecology Datasets==

'''ENVIRONMENTAL DATA'''

::* [[ ECMWF|Weather Forecast Data (ECMWF)]]
::* [[ Elevation|Elevation Data ]]

'''FLYSAFE'''

:Bird data created from weather radar data and bird migration models. Financed by the Royal Netherlands Air Force.

::* [[Connecting to the Flysafe database]]
::* [[GPS|Flysafe GPS Data Model]]
::* [[ RADAR|Radar Data Model]]
::* [[ Wind profile|Wind Profile Data Model]]
::* [[ Fino1|Fino1 Data Model]]

==Other e-Ecology Services==
[[GPS tracks|FlySafe GPS Bird Track Generator]]: Produce kmz files of ARGOS/GPS bird tracking data from the FlySafe project to be viewed in Google Earth.

[[ Administration]]: Information about user rights and data processing tasks.

[[ Bookkeeping]]: Description of tables related to FlySafe data processing and archiving.

[[ ToDo ]]

ECMWF

2015-01-29T14:24:24Z

Judy: /* Helpful Functions & Queries */

== ECMWF ==

Datamodel for the tables in the ECMWF schema.

Data is created by the ECMWF and retrieved from the KNMI ftp site as GRIB files. The original GRIB files can also be retrieved from the flysafe server from :

[https://flysafe-pps.grid.sara.nl/archive/knmi/ecmwf/ ECMWF Data Archive]

More information about the parameters in these GRIB files can be retrieved from the [http://www.ecmwf.int/services/dissemination/3.1/Products_in_GRIB_code.html ECMWF website].

== INVARIABLE ==

The INVARIABLE table contains data that is invariant over time.

{|border=1
|'''LATITUDE''' || degrees * 100 to make this a integer
|-
|'''LONGITUDE''' || degrees * 100 to make this a integer
|-
|Z || 129 Z Geopotential surface/model orography
|-
|LSM || 172 LSM Land sea mask
|-
|LOCATION || Latitude/longitude as GIS location
|}

The GIS LOCATION is only added to this table. This is because always the same values for latitude and longitude are used and then if the GIS location is required for the ANALYSIS or FORECAST tables it can be retrieved by joining to this table on the LATITUDE, LONGUTUDE fields.

== FORECAST and UVA_FORECAST ==

The FORECAST table contains predicted values in steps of 3 hours till 72 hours ahead. The MODELTS is the timestamp for which this prediction applies. Two times a day new Forecast GRIB files will arrive. The newer FORECAST data that arrives for the same timestamp will overwrite the older data.

The FORECAST (and UVA_FORECAST) table itself is a master table that does not contain any data. The inherited tables
FORECAST<YEAR><MONTH> contain the data, but the master FORECAST or UVA_FORECAST table can be used to SELECT the data.

Data till 2009-09 is owned by the UvA and can be accessed by anybody with ECMWF_READ rights by using the FORECAST_UVA table. The FORECAST table itself also references data after 2009-09 that is owned by RNLAF for which special access rights ECMWF_RNLAF_READ are required.

{|border=1
|'''MODELTS'''||Time for which this forecast applies = model run + step hours
|-
|'''LATITUDE'''||degrees * 100 to make this a integer
|-
|'''LONGITUDE'''||degrees * 100 to make this a integer
|-
|STEP||model step (smaller is more recent forecast)
|-
|SSRD||169 SSRD surface solar radiation downwards accumulated
|-
|TP||228 TP total precipitation accumulated
|-
|SP||134 SP surface pressure
|-
|BLH||159 BLH boundary layer height
|-
|TCC||164 TCC total cloud coverage
|-
|U10||165 10U 10 meters wind U-component
|-
|V10||166 10V 10 meters wind V-component
|-
|T2||167 2T 2 metres air temperature
|-
|D2||168 2D 2 metres dew point temperature
|-
|LCC||186 LCC low cloud coverage
|-
|''Pressure level 1000''||
|-
|T_1000||130 T temperature
|-
|U_1000||131 U U-wind component
|-
|V_1000||132 V V-wind component
|-
|GH_1000||156 GH geopotential height
|-
|R_1000||157 R relative humidity
|-
|''Pressure level 925''||
|-
|T_925 real||130 T temperature
|-
|U_925 real||131 U U-wind component
|-
|V_925 real||132 V V-wind component
|-
|GH_925 real||156 GH geopotential height
|-
|R_925 real||157 R relative humidity
|-
|''Pressure level 850''||
|-
|T_850||130 T temperature
|-
|U_850 real||131 U U-wind component
|-
|V_850 real||132 V V-wind component
|-
|GH_850 real||156 GH geopotential height
|-
|R_850 real||157 R relative humidity
|-
|''Pressure level 700''||
|-
|T_700 real||130 T temperature
|-
|U_700 real||131 U U-wind component
|-
|V_700 real||132 V V-wind component
|-
|GH_700 real||156 GH geopotential height
|-
|R_700 real||157 R relative humidity
|}

== ANALYSIS and UVA_ANALYSIS ==

The ANALYSIS and UVA_ANALYSIS table contains values from the Analysis GRIB files at 18, 00, 06 and 12 UTC

Data till 2009-09 is owned by the UvA and can be accessed by anybody with ECMWF_READ rights by using the ANALYSIS_UVA table. The ANALYSIS table itself also references data after 2009-09 that is owned by RNLAF for which special access rights ECMWF_RNLAF_READ are required.

{|border=1
|'''MODELTS'''||Time for which this forecast applies = model run
|-
|'''LATITUDE'''||degrees * 100 to make this a integer
|-
|'''LONGITUDE'''||degrees * 100 to make this a integer
|-
|SP||134 SP surface pressure
|-
|TCC||164 TCC total cloud coverage
|-
|U10||165 10U 10 meters wind U-component
|-
|V10||166 10V 10 meters wind V-component
|-
|T2||167 2T 2 metres air temperature
|-
|D2||168 2D 2 metres dew point temperature
|-
|LCC||186 LCC low cloud coverage
|-
|''Pressure level 1000''||
|-
|T_1000||130 T temperature
|-
|U_1000||131 U U-wind component
|-
|V_1000||132 V V-wind component
|-
|GH_1000||156 GH geopotential height
|-
|R_1000||157 R relative humidity
|-
|Pressure level 925||
|-
|T_925||130 T temperature
|-
|U_925||131 U U-wind component
|-
|V_925||132 V V-wind component
|-
|GH_925||156 GH geopotential height
|-
|R_925||157 R relative humidity
|-
|Pressure level 850||
|-
|T_850||130 T temperature
|-
|U_850||131 U U-wind component
|-
|V_850||132 V V-wind component
|-
|GH_850||156 GH geopotential height
|-
|R_850||157 R relative humidity
|-
|Pressure level 700||
|-
|T_700||130 T temperature
|-
|U_700||131 U U-wind component
|-
|V_700||132 V V-wind component
|-
|GH_700||156 GH geopotential height
|-
|R_700||157 R relative humidity
|}

== COMBINED ==

If you want to use Analysis data if available for a timestimp but otherwise Forecast, the COMBINED VIEW can be used. If a ANALYSIS value is available for a timestamp and latitude/longitude the ANALYSIS value will be used, otherwise the FORECAST value is given.

== Historical data ==

In addition ongoing process of adding ECMWF data since November 11 2007, a historical data set has been loaded. That historical dataset is stored in additional inherited tables FORECASTHISTORY and
ANALYSISHISTORY which inherit from FORECAST and ANALYSIS respectively.

The FORECASTHISTORY does not always contain TP (228 TP total precipitation accumulated). Instead of this it contains the following values :

{|border=1
|LSP||142 LSP Large scale precipitation
|-
|CP||143 CP Convective precipitation
|}

The historical dat sets span the following periods and locations

{|border=1
|'''Start'''||'''End'''||'''Area'''||'''Grid'''
|-
|2007-01-01||2007-11-15||62degN 10degW 335degN 20degE||0.25x0.25deg
|-
|2003-02-15||2003-06-15||62degN 10degW 335degN 20degE||0.5x0.5deg
|-
|2005-01-01||2006-12-31||62degN 10degW 335degN 20degE||0.5x0.5deg
|}

== Helpful Functions & Queries ==

The following functions have been defined in Postgres to select the nearest gridpoint in spacetime for the ECMWF analysis and forecast tables.

These are '''ecmwf.forecast_nearest''' and '''ecmwf.analysis_nearest''' and they take a latitude, longitude and timestamp as parameter.

They can be used in the following way, which will return all the fields in the relevant ECMWF data:

SELECT * FROM ecmwf.forecast_nearest( 52.35983, 5.615, timestamp '2008-05-19 13:34:00');

SELECT * FROM ecmwf.analysis_nearest( 52.35983, 5.615, timestamp '2008-05-19 13:34:00');

Note that the model latitude and model longitude are returned as integers and multiples of 100. To return latititude and longitude as a decimal (for example to compare to the latitude and longitude of your GPS data) you need to first convert these fields to decimals:

SELECT f.latitude/100::decimal, f.longitude/100::decimal
FROM ecmwf.forecast_nearest( 52.35983, 5.615, timestamp '2008-05-19 13:34:00') f;

These function can for example be used in a join as follows :

SELECT g.ptt_id
, g.date_time
, g.latitude
, g.longitude
, g.speed
, g.course
, g.altitude
, (ecmwf.forecast_nearest( g.latitude, g.longitude,g.date_time)).*
FROM gps.gps_tracking_data g
, gps.individual i
, gps.track_session t
WHERE g.ptt_id = t.ptt_id
AND g.date_time >= t.start_date
AND i.ring_number = t.ring_number
AND g.parser_qc = 1
AND g.ptt_id = 41745
AND g.latitude <> 0
ORDER BY g.date_time;

Although these functions were converted to PLPgSQL to speed them up, a faster ways to retrieve the same data is with (thanks to Floris Sluiters):

SELECT ptt_id, date_time, latitude, longitude, speed, course, altitude, (x::ecmwf.forecast).*
FROM (SELECT
g.*
,((SELECT f
FROM ecmwf.forecast f
WHERE f.latitude = round(g.latitude * 100 / 25)::integer * 25
AND f.longitude = round(g.longitude * 100 / 25)::integer * 25
AND f.modelts = TIMESTAMP 'epoch' + round(EXTRACT(EPOCH FROM g.date_time)
/ (60*60*3))::integer * INTERVAL '3 hours' limit 1)::ecmwf.forecast) as x
FROM gps.gps_tracking_data g
, gps.individual i
, gps.track_session t
WHERE g.ptt_id = t.ptt_id
AND g.date_time >= t.start_date
AND i.ring_number = t.ring_number
AND g.parser_qc = 1
AND g.ptt_id = 41745
AND g.latitude <> 0
ORDER BY g.date_time) t

Below is example query that retrieves data for a model grid point closest to user provided coordinates and for one model time stamp (eg. 12:00) for a series of consecutive days

SELECT f.modelts, f.latitude, f.longitude, f.blh
FROM ecmwf.forecast as f
WHERE f.latitude = round(42.718104*100/25)::integer*25 AND f.longitude = round(11.517727*100/25)::integer*25 and
f.modelts >= '2007-08-01 12:00' and f.modelts <= '2008-08-01 12:00' and date_part('hour', f.modelts) = 12
ORDER BY f.modelts

=== Interpolation ===

The following functions are added so that simple linear interpolation for a specific point in space and time can be done in Postgres.

These are '''ecmwf.forecast_interpolate''' and '''ecmwf.analysis_interpolate''' and they take a latitude, longitude and timestamp as parameter.

They can be used in the same way:

select * from ecmwf.forecast_interpolate( 52.35983, 5.615, timestamp '2008-05-19 13:34:00');

select * from ecmwf.analysis_interpolate( 52.35983, 5.615, timestamp '2008-05-19 13:34:00');

And they can also be used in joins as above. These functions select the 8 surrounding points in latitude, longitude and date_time and interpolate linear in latitude, longitude and date_time successively. The '''interpolate''' functions are much slower then the '''nearest''' functions.

Useful Queries

2015-01-19T19:02:32Z

Judy: /* Total number of measurements per tag */

Useful Queries

2015-01-19T14:31:33Z

Judy: /* Re-sample tracking data to select data with a predefined interval between fixes */

Useful Queries

2015-01-19T14:30:38Z

Judy: /* Select tracking data and distinguish between distinct GPS deployments */

= UvAGPS functions =

== get_uvagps_track_speed ==

Compute distance (meters), time-interval (time stamp hh:mm:ss), speed (meters/second) and heading (degrees) for a series of subsequent tracking entries for a specific device_info_serial.
For the first entry in the sequence these values will always be missing because they are computed as the difference between subsequent rows.

Usage examples :

<pre>
select * from gps.get_uvagps_track_speed(<device_info_serial>);

select * from gps.get_uvagps_track_speed(119, '2008-01-01 00:00:00', '2010-01-01 00:00:00');

select t.*, a.distance, a.interval, a.speed, a.direction
from gps.get_uvagps_track_speed(119, '2008-01-01 00:00:00', '2010-01-01 00:00:00') a
join gps.ee_tracking_speed_limited t using(device_info_serial, date_time);
</pre>

Distances and heading are calculated with spherical correction with the Haversine formula as described [http://http://www.movable-type.co.uk/scripts/latlong.html here].
The heading is the initial heading. PostGIS does not calculate the heading, so therefore we don't use PostGIS.

In these functions the records that have been flagged as invalid by setting the '''userflag''' to 1 are not taken into account and discarded.
If speed calculations are required for all records an additional boolean argument '''get raw data''' can be given. i.e.

<pre>
select * from gps.get_uvagps_track_speed(119, '2008-01-01 00:00:00', '2010-01-01 00:00:00', true);

select * from gps.get_uvagps_track_speed(119, true);
</pre>

== get_uvagps_track_distance ==

Compute distance (meters) for UvAGPS tracks to arbitrary point in lat/long or geometry location

Example usage :

<pre>
select * from gps.get_uvagps_track_distance(119, 52.1897017, 6.1996606);

select * from gps.get_uvagps_track_distance(119, (select location from gps.ee_tracking_speed_limited where device_info_serial = 119 and date_time = '2009-05-07 19:55:08'));

</pre>

Here the distance is calculated with the PostGIS distance_sphere function which uses spherical correction.

== get_uvagps_track_distance_direction ==

Compute distance (meters) and direction (degrees) for UvAGPS tracks from arbitrary point in lat/long or geometry location

Example usage:

<pre>
select * from gps.get_uvagps_track_distance_direction(119, 52.1897017, 6.1996606);
</pre>

Distances and direction are calculated with spherical correction with the Haversine formula as described [http://http://www.movable-type.co.uk/scripts/latlong.html here]. The direction is the direction from the given point.

In this function the records where the '''userflag''' has been set are discarded. To see these records a additional boolean argument for '''show raw data''' should be added like :

<pre>
select * from gps.get_uvagps_track_distance_direction(119, 52.1897017, 6.1996606, true);
</pre>
= Filtering tracking data =

== Select tracking data from the beginning of a track session ==

Select only those points that were collected during a valid track session (i.e. data that begins from the start_date of a track session) including 2D GPS speed. Filter our points where no GPS fix is returned (latitude is not null) and where a user has flagged the data as inappropriate (user_flag <> 1):

SELECT t.device_info_serial, t.date_time, t.latitude, t.longitude, t.altitude, t.temperature, t.speed, t.userflag
FROM gps.ee_tracking_speed_limited t, gps.ee_track_session_limited s
WHERE t.device_info_serial = s.device_info_serial AND t.date_time >= s.start_date AND t.device_info_serial = '''<ID>'''
AND t.latitude IS NOT NULL AND t.userflag <> 1

== Select tracking data and distinguish between distinct GPS deployments ==
In this query you will select the tracking data related to each deployment of a GPS logger or each time an individual bird was tracked. This query is slightly more complex than the query above. If you have used the same logger on more than one individual bird, or have tracked the same individual with more than one logger, this query will enable you to make a distinction between these tracking events by showing both GPS logger id (device_info_serial) and a unique identifier for your individual (eg ring_number or colour_ring). Furthermore, as above, this will only select tracking data from the start and end of a particular track session.

SELECT t.device_info_serial, t.date_time, t.latitude, t.longitude, t.altitude, t.speed_2d, i.colour_ring, s.track_session_id, s.key_name
FROM gps.ee_track_session_limited s
JOIN gps.ee_individual_limited i
ON s.individual_id = i.individual_id
JOIN gps.ee_tracking_speed_limited t
ON t.device_info_serial = s.device_info_serial
AND t.date_time BETWEEN s.start_date AND s.end_date
WHERE s.key_name = ''''<Project name>'''' AND t.userflag = 0
ORDER BY t.device_info_serial, t.date_time

HINT: WHERE s.key_name = 'HG_TEXEL'

== Re-sample tracking data to select data with a predefined interval between fixes ==

The following query provides an example of how to select data points with a predefined interval between each point and not lower. In the following example, a 5 minute interval is selected. In this example all measurements that are taken between each 5 minute interval are left out of the resampled data.

Commented query with examples. Statements followed by -- are comments and are not parsed by SQL.

SELECT *
FROM gps.ee_tracking_speed_limited
JOIN
(
SELECT
device_info_serial,
-- return oldest date_time for each interval
MIN(date_time) date_time,
-- each 5 minutes has a unique identifier
FLOOR(EXTRACT(EPOCH FROM date_time)/(5*60)) intervalid
FROM gps.ee_tracking_speed_limited
GROUP BY device_info_serial, intervalid
) b
USING (device_info_serial, date_time)
WHERE device_info_serial = 355 AND date_time BETWEEN '2010-06-27' AND '2010-06-29'
ORDER BY date_time

Uncommented query, date_time and device_info_serial need to be filled in. Interval is 5 minutes and can be changed.

SELECT *
FROM gps.ee_tracking_speed_limited
JOIN
(
SELECT
device_info_serial,
MIN(date_time) date_time,
FLOOR(EXTRACT(EPOCH FROM date_time)/(5*60)) intervalid
FROM gps.ee_tracking_speed_limited
GROUP BY device_info_serial, intervalid
) b
USING (device_info_serial, date_time)
WHERE device_info_serial = <ID> AND date_time BETWEEN 'yyyy-mm-dd' AND 'yyyy-mm-dd'
ORDER BY date_time

= Calculating summary statistics =
== Total distance ==
A query to determine the total distance a bird has flown is :

select sum(distance)/1000 as total_distance
from gps.get_uvagps_track_speed('''<ID>''');

or for a fixed time period :

select sum(distance)/1000 as total_distance
from gps.get_uvagps_track_speed('''<ID>''', '2010-01-01 00:00:00', '2011-01-01 00:00:00');

== Total number of measurements per tag ==

Count the total number of measurements for each device using the GROUP BY clause:

select device_info_serial, count(*)
from gps.ee_tracking_speed_limited
group by device_info_serial
ORDER BY device_info_serial ASC;

If you want to count the number of measurements with valid GPS fixes you can add a where clause :

select device_info_serial, count(*)
from gps.ee_tracking_speed_limited
where latitude is not null
group by device_info_serial
ORDER BY device_info_serial ASC;

Useful Queries

2015-01-19T14:28:57Z

Judy: /* Re-sample tracking data to select data with a predefined interval between fixes */

= UvAGPS functions =

== get_uvagps_track_speed ==

Compute distance (meters), time-interval (time stamp hh:mm:ss), speed (meters/second) and heading (degrees) for a series of subsequent tracking entries for a specific device_info_serial.
For the first entry in the sequence these values will always be missing because they are computed as the difference between subsequent rows.

Usage examples :

<pre>
select * from gps.get_uvagps_track_speed(<device_info_serial>);

select * from gps.get_uvagps_track_speed(119, '2008-01-01 00:00:00', '2010-01-01 00:00:00');

select t.*, a.distance, a.interval, a.speed, a.direction
from gps.get_uvagps_track_speed(119, '2008-01-01 00:00:00', '2010-01-01 00:00:00') a
join gps.ee_tracking_speed_limited t using(device_info_serial, date_time);
</pre>

Distances and heading are calculated with spherical correction with the Haversine formula as described [http://http://www.movable-type.co.uk/scripts/latlong.html here].
The heading is the initial heading. PostGIS does not calculate the heading, so therefore we don't use PostGIS.

In these functions the records that have been flagged as invalid by setting the '''userflag''' to 1 are not taken into account and discarded.
If speed calculations are required for all records an additional boolean argument '''get raw data''' can be given. i.e.

<pre>
select * from gps.get_uvagps_track_speed(119, '2008-01-01 00:00:00', '2010-01-01 00:00:00', true);

select * from gps.get_uvagps_track_speed(119, true);
</pre>

== get_uvagps_track_distance ==

Compute distance (meters) for UvAGPS tracks to arbitrary point in lat/long or geometry location

Example usage :

<pre>
select * from gps.get_uvagps_track_distance(119, 52.1897017, 6.1996606);

select * from gps.get_uvagps_track_distance(119, (select location from gps.ee_tracking_speed_limited where device_info_serial = 119 and date_time = '2009-05-07 19:55:08'));

</pre>

Here the distance is calculated with the PostGIS distance_sphere function which uses spherical correction.

== get_uvagps_track_distance_direction ==

Compute distance (meters) and direction (degrees) for UvAGPS tracks from arbitrary point in lat/long or geometry location

Example usage:

<pre>
select * from gps.get_uvagps_track_distance_direction(119, 52.1897017, 6.1996606);
</pre>

Distances and direction are calculated with spherical correction with the Haversine formula as described [http://http://www.movable-type.co.uk/scripts/latlong.html here]. The direction is the direction from the given point.

In this function the records where the '''userflag''' has been set are discarded. To see these records a additional boolean argument for '''show raw data''' should be added like :

<pre>
select * from gps.get_uvagps_track_distance_direction(119, 52.1897017, 6.1996606, true);
</pre>
= Filtering tracking data =

== Select tracking data from the beginning of a track session ==

Select only those points that were collected during a valid track session (i.e. data that begins from the start_date of a track session) including 2D GPS speed. Filter our points where no GPS fix is returned (latitude is not null) and where a user has flagged the data as inappropriate (user_flag <> 1):

SELECT t.device_info_serial, t.date_time, t.latitude, t.longitude, t.altitude, t.temperature, t.speed, t.userflag
FROM gps.ee_tracking_speed_limited t, gps.ee_track_session_limited s
WHERE t.device_info_serial = s.device_info_serial AND t.date_time >= s.start_date AND t.device_info_serial = '''<ID>'''
AND t.latitude IS NOT NULL AND t.userflag <> 1

== Select tracking data and distinguish between distinct GPS deployments ==
In this query you will select the tracking data related to each deployment of a GPS logger or each time an individual bird was tracked. This query is slightly more complex than the query above. If you have used the same logger on more than one individual bird, or have tracked the same individual with more than one logger, this query will enable you to make a distinction between these tracking events by showing both GPS logger id (device_info_serial) and a unique identifier for your individual (eg ring_number or colour_ring). Furthermore, as above, this will only select tracking data from the start and end of a particular track session.

SELECT t.device_info_serial, t.date_time, t.latitude, t.longitude, t.altitude, t.speed_2d, i.colour_ring, s.track_session_id, s.key_name
FROM gps.ee_track_session_limited s
JOIN gps.ee_individual_limited i
ON s.individual_id = i.individual_id
JOIN gps.ee_tracking_speed_limited t
ON t.device_info_serial = s.device_info_serial
AND t.date_time BETWEEN s.start_date AND s.end_date
WHERE s.key_name = '<Project name>' AND t.userflag = 0
ORDER BY t.device_info_serial, t.date_time

HINT: WHERE s.key_name = 'HG_TEXEL'

== Re-sample tracking data to select data with a predefined interval between fixes ==

The following query provides an example of how to select data points with a predefined interval between each point and not lower. In the following example, a 5 minute interval is selected. In this example all measurements that are taken between each 5 minute interval are left out of the resampled data.

Commented query with examples. Statements followed by -- are comments and are not parsed by SQL.

SELECT *
FROM gps.ee_tracking_speed_limited
JOIN
(
SELECT
device_info_serial,
-- return oldest date_time for each interval
MIN(date_time) date_time,
-- each 5 minutes has a unique identifier
FLOOR(EXTRACT(EPOCH FROM date_time)/(5*60)) intervalid
FROM gps.ee_tracking_speed_limited
GROUP BY device_info_serial, intervalid
) b
USING (device_info_serial, date_time)
WHERE device_info_serial = 355 AND date_time BETWEEN '2010-06-27' AND '2010-06-29'
ORDER BY date_time

Uncommented query, date_time and device_info_serial need to be filled in. Interval is 5 minutes and can be changed.

SELECT *
FROM gps.ee_tracking_speed_limited
JOIN
(
SELECT
device_info_serial,
MIN(date_time) date_time,
FLOOR(EXTRACT(EPOCH FROM date_time)/(5*60)) intervalid
FROM gps.ee_tracking_speed_limited
GROUP BY device_info_serial, intervalid
) b
USING (device_info_serial, date_time)
WHERE device_info_serial = <ID> AND date_time BETWEEN 'yyyy-mm-dd' AND 'yyyy-mm-dd'
ORDER BY date_time

= Calculating summary statistics =
== Total distance ==
A query to determine the total distance a bird has flown is :

select sum(distance)/1000 as total_distance
from gps.get_uvagps_track_speed('''<ID>''');

or for a fixed time period :

select sum(distance)/1000 as total_distance
from gps.get_uvagps_track_speed('''<ID>''', '2010-01-01 00:00:00', '2011-01-01 00:00:00');

== Total number of measurements per tag ==

Count the total number of measurements for each device using the GROUP BY clause:

select device_info_serial, count(*)
from gps.ee_tracking_speed_limited
group by device_info_serial
ORDER BY device_info_serial ASC;

If you want to count the number of measurements with valid GPS fixes you can add a where clause :

select device_info_serial, count(*)
from gps.ee_tracking_speed_limited
where latitude is not null
group by device_info_serial
ORDER BY device_info_serial ASC;

Useful Queries

2015-01-19T12:58:52Z

Judy: /* Filtering tracking data */

= UvAGPS functions =

== get_uvagps_track_speed ==

Compute distance (meters), time-interval (time stamp hh:mm:ss), speed (meters/second) and heading (degrees) for a series of subsequent tracking entries for a specific device_info_serial.
For the first entry in the sequence these values will always be missing because they are computed as the difference between subsequent rows.

Usage examples :

<pre>
select * from gps.get_uvagps_track_speed(<device_info_serial>);

select * from gps.get_uvagps_track_speed(119, '2008-01-01 00:00:00', '2010-01-01 00:00:00');

select t.*, a.distance, a.interval, a.speed, a.direction
from gps.get_uvagps_track_speed(119, '2008-01-01 00:00:00', '2010-01-01 00:00:00') a
join gps.ee_tracking_speed_limited t using(device_info_serial, date_time);
</pre>

Distances and heading are calculated with spherical correction with the Haversine formula as described [http://http://www.movable-type.co.uk/scripts/latlong.html here].
The heading is the initial heading. PostGIS does not calculate the heading, so therefore we don't use PostGIS.

In these functions the records that have been flagged as invalid by setting the '''userflag''' to 1 are not taken into account and discarded.
If speed calculations are required for all records an additional boolean argument '''get raw data''' can be given. i.e.

<pre>
select * from gps.get_uvagps_track_speed(119, '2008-01-01 00:00:00', '2010-01-01 00:00:00', true);

select * from gps.get_uvagps_track_speed(119, true);
</pre>

== get_uvagps_track_distance ==

Compute distance (meters) for UvAGPS tracks to arbitrary point in lat/long or geometry location

Example usage :

<pre>
select * from gps.get_uvagps_track_distance(119, 52.1897017, 6.1996606);

select * from gps.get_uvagps_track_distance(119, (select location from gps.ee_tracking_speed_limited where device_info_serial = 119 and date_time = '2009-05-07 19:55:08'));

</pre>

Here the distance is calculated with the PostGIS distance_sphere function which uses spherical correction.

== get_uvagps_track_distance_direction ==

Compute distance (meters) and direction (degrees) for UvAGPS tracks from arbitrary point in lat/long or geometry location

Example usage:

<pre>
select * from gps.get_uvagps_track_distance_direction(119, 52.1897017, 6.1996606);
</pre>

Distances and direction are calculated with spherical correction with the Haversine formula as described [http://http://www.movable-type.co.uk/scripts/latlong.html here]. The direction is the direction from the given point.

In this function the records where the '''userflag''' has been set are discarded. To see these records a additional boolean argument for '''show raw data''' should be added like :

<pre>
select * from gps.get_uvagps_track_distance_direction(119, 52.1897017, 6.1996606, true);
</pre>
= Filtering tracking data =

== Select tracking data from the beginning of a track session ==

Select only those points that were collected during a valid track session (i.e. data that begins from the start_date of a track session) including 2D GPS speed. Filter our points where no GPS fix is returned (latitude is not null) and where a user has flagged the data as inappropriate (user_flag <> 1):

SELECT t.device_info_serial, t.date_time, t.latitude, t.longitude, t.altitude, t.temperature, t.speed, t.userflag
FROM gps.ee_tracking_speed_limited t, gps.ee_track_session_limited s
WHERE t.device_info_serial = s.device_info_serial AND t.date_time >= s.start_date AND t.device_info_serial = '''<ID>'''
AND t.latitude IS NOT NULL AND t.userflag <> 1

== Select tracking data and distinguish between distinct GPS deployments ==
In this query you will select the tracking data related to each deployment of a GPS logger or each time an individual bird was tracked. This query is slightly more complex than the query above. If you have used the same logger on more than one individual bird, or have tracked the same individual with more than one logger, this query will enable you to make a distinction between these tracking events by showing both GPS logger id (device_info_serial) and a unique identifier for your individual (eg ring_number or colour_ring). Furthermore, as above, this will only select tracking data from the start and end of a particular track session.

SELECT t.device_info_serial, t.date_time, t.latitude, t.longitude, t.altitude, t.speed_2d, i.colour_ring, s.track_session_id, s.key_name
FROM gps.ee_track_session_limited s
JOIN gps.ee_individual_limited i
ON s.individual_id = i.individual_id
JOIN gps.ee_tracking_speed_limited t
ON t.device_info_serial = s.device_info_serial
AND t.date_time BETWEEN s.start_date AND s.end_date
WHERE s.key_name = '<Project name>' AND t.userflag = 0
ORDER BY t.device_info_serial, t.date_time

HINT: WHERE s.key_name = 'HG_TEXEL'

== Re-sample tracking data to select data with a predefined interval between fixes ==

The following query provides an example of how to select data points with a predefined interval between each point and not lower. In the following example, a 5 minute interval is selected. In this example all measurements that are taken between each 5 minute interval are left out of the resampled data.

Commented query with examples. Statements followed by -- are comments and are not parsed by SQL.

SELECT *
FROM gps.ee_tracking_speed_limited
JOIN
(
SELECT
device_info_serial,
-- return oldest date_time for each interval
MIN(date_time) date_time,
-- each 5 minutes has a unique identifier
FLOOR(EXTRACT(EPOCH FROM date_time)/(5*60)) intervalid
FROM gps.ee_tracking_speed_limited
GROUP BY device_info_serial, intervalid
) b
USING (device_info_serial, date_time)
WHERE device_info_serial = 355 AND date_time BETWEEN '2010-06-27' AND '2010-06-29'
ORDER BY date_time

Uncommented query, date_time and device_info_serial need to be filled in. Interval is 5 minutes and can be changed.

SELECT *
FROM gps.ee_tracking_speed_limited
JOIN
(
SELECT
device_info_serial,
MIN(date_time) date_time,
FLOOR(EXTRACT(EPOCH FROM date_time)/(5*60)) intervalid
FROM gps.ee_tracking_speed_limited
GROUP BY device_info_serial, intervalid
) b
USING (device_info_serial, date_time)
WHERE device_info_serial = <ID> AND date_time BETWEEN 'yyyy-mm-dd' AND 'yyyy-mm-dd'
ORDER BY date_time

= Calculating summary statistics =
== Total distance ==
A query to determine the total distance a bird has flown is :

select sum(distance)/1000 as total_distance
from gps.get_uvagps_track_speed('''<ID>''');

or for a fixed time period :

select sum(distance)/1000 as total_distance
from gps.get_uvagps_track_speed('''<ID>''', '2010-01-01 00:00:00', '2011-01-01 00:00:00');

== Total number of measurements per tag ==

Count the total number of measurements for each device using the GROUP BY clause:

select device_info_serial, count(*)
from gps.ee_tracking_speed_limited
group by device_info_serial
ORDER BY device_info_serial ASC;

If you want to count the number of measurements with valid GPS fixes you can add a where clause :

select device_info_serial, count(*)
from gps.ee_tracking_speed_limited
where latitude is not null
group by device_info_serial
ORDER BY device_info_serial ASC;

Useful Queries

2015-01-19T12:06:13Z

Judy: /* Calculating summary statistics */

Useful Queries

2015-01-19T12:05:31Z

Judy: /* UvAGPS functions */

Useful Queries

2015-01-14T09:05:13Z

Judy: /* How to separate birds or tag deployments when querying data where the same GPS logger has been used more than once or the same individual has been tracked with more than one logger */

Useful Queries

2015-01-14T09:03:42Z

Judy: /* How to separate birds or tag deployments when querying data where the same GPS logger has been used more than once or the same individual has been tracked with more than one logger */

Useful Queries

2015-01-14T08:57:23Z

Judy: /* get_uvagps_track_distance */

Useful Queries

2015-01-14T08:56:41Z

Judy: /* get_uvagps_track_speed */

Useful Queries

2015-01-14T08:55:41Z

Judy: /* get_uvagps_track_speed */

Useful Queries

2015-01-09T11:23:09Z

Judy: /* How to separate birds or tag deployments when querying data where the same GPS logger has been used more than once or the same individual has been tracked with more than one logger */

Useful Queries

2015-01-09T11:21:57Z

Judy: /* How to separate birds or tag deployments when querying data where the same GPS logger has been used more than once or the same individual has been tracked with more than on logger */

E-Ecology Documentation and Guidance

2014-12-18T20:00:10Z

Judy: /* Changes in the Database in 2014 */

The e-Ecology Virtual Laboratories represent a collaborative effort between the University of Amsterdam Institute for Biodiversity and Ecosystem Dynamics and SARA Computing to manage bird tracking data and develop tools for data exploration, analysis and visualization. This wiki is designed to facilitate access to the data and tools within the Virtual Laboratories and to provide a forum for collaboration.

== UvA-BiTS data ==

:Bird data collected with the UvA-BiTS Bird Tracking System

Note: We are continually working on improving UvA-BiTS services, including data processing and Virtual Lab functionality. In some cases, this has enabled us to extract more information from old GPS log files which is then incorporated into the database. These changes may result in slight differences when you reprocess historic data.

::* [[ DB_Model_2014 | e-Ecology data model]]
::* [[ UvAGPS|UvA-BiTS Data Model]] - OLD
::* [[Connecting to the UvA-BiTS database]]
::* [[Useful Queries]]
::* [[Queries for linking bird tracks to wind and flow data]]

==UvA-BiTS Services==

* Visualization Tools
:: [[KMZ Generator2]]: 
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth.
:: [[Calendar]]: 
:::Calendar overview with daily statistics of GPS-tracker
:: [[GPX]]: 
:::This tool produces GPX files from UvA-BiTS tracking data for dynamic [http://www.doarama.com Doarama] visualizations.
:: [[BirdView]]: 
:::Access meterological and radar data and overlay with GPS bird tracking data.
:: [[BirdSim]]: 
:::More dynamic version of BirdView that allows the user to scroll through weather data in time and space.
:: [[KMZ Generator|UvA-BiTS KMZ Generator]]: 
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth.

* [[Dashboard]]: These web pages allow UvA-BiTS users to get an overview of the raw data files of GPS bird tracking devices.

* [[ Annotation tables ]] for uploading GPS annotation data into your own schema. This service is currently not available.

== Changes in the Database in 2014 ==
'''Important'''
In the summer of 2014 we have '''re-arranged the administrative''' part of UvA-BiTS (and e-Ecology) in the database. End users may have to adapt their programs the following way:
:* The '''uva_*_limited''' views that used to exist in the gps schema are gone. Most of them should have been replaced with an equivalent '''ee_*_limited counterpart'''.
:* The new ee_*_limited views show a superset of the same information as before, but enhanced with new project information.
:* Some '''column names''' in the ee_*_limited views may differ from the previous views.
:* There are new limited views:
:** gps.ee_'''project'''_limited: shows your projects' information
:** gps.ee_'''shared'''_*_limited: set of data you are able to see related to trackers that a project is sharing with a project you belong to
:* Special attention: gps.uva_device_limited is now replaced by '''gps.ee_tracker_limited'''. For each tracker, you now see which project provides you access to that tracker.
:* Also, '''gps.uva_tracking_limited''' is gone. Instead of that we have gps.ee_tracker_speed_limited, which adds speed columns to the original data. This amounts to a merger of gps.ee_tracking_limited, gps.ee_tracking_speed_3d_limited and gps.ee_tracking_speed_limited.

==Other e-Ecology Datasets==

'''ENVIRONMENTAL DATA'''

::* [[ ECMWF|Weather Forecast Data (ECMWF)]]
::* [[ Elevation|Elevation Data ]]

'''FLYSAFE'''

:Bird data created from weather radar data and bird migration models. Financed by the Royal Netherlands Air Force.

::* [[Connecting to the Flysafe database]]
::* [[GPS|Flysafe GPS Data Model]]
::* [[ RADAR|Radar Data Model]]
::* [[ Wind profile|Wind Profile Data Model]]
::* [[ Fino1|Fino1 Data Model]]

==Other e-Ecology Services==
[[GPS tracks|FlySafe GPS Bird Track Generator]]: Produce kmz files of ARGOS/GPS bird tracking data from the FlySafe project to be viewed in Google Earth.

[[ Administration]]: Information about user rights and data processing tasks.

[[ Bookkeeping]]: Description of tables related to FlySafe data processing and archiving.

[[ ToDo ]]

E-Ecology Documentation and Guidance

2014-12-18T19:59:43Z

Judy: /* Other e-Ecology Datasets */

The e-Ecology Virtual Laboratories represent a collaborative effort between the University of Amsterdam Institute for Biodiversity and Ecosystem Dynamics and SARA Computing to manage bird tracking data and develop tools for data exploration, analysis and visualization. This wiki is designed to facilitate access to the data and tools within the Virtual Laboratories and to provide a forum for collaboration.

== Changes in the Database in 2014 ==
'''Important'''
In the summer of 2014 we have '''re-arranged the administrative''' part of UvA-BiTS (and e-Ecology) in the database. End users may have to adapt their programs the following way:
:* The '''uva_*_limited''' views that used to exist in the gps schema are gone. Most of them should have been replaced with an equivalent '''ee_*_limited counterpart'''.
:* The new ee_*_limited views show a superset of the same information as before, but enhanced with new project information.
:* Some '''column names''' in the ee_*_limited views may differ from the previous views.
:* There are new limited views:
:** gps.ee_'''project'''_limited: shows your projects' information
:** gps.ee_'''shared'''_*_limited: set of data you are able to see related to trackers that a project is sharing with a project you belong to
:* Special attention: gps.uva_device_limited is now replaced by '''gps.ee_tracker_limited'''. For each tracker, you now see which project provides you access to that tracker.
:* Also, '''gps.uva_tracking_limited''' is gone. Instead of that we have gps.ee_tracker_speed_limited, which adds speed columns to the original data. This amounts to a merger of gps.ee_tracking_limited, gps.ee_tracking_speed_3d_limited and gps.ee_tracking_speed_limited.

== UvA-BiTS data ==

:Bird data collected with the UvA-BiTS Bird Tracking System

Note: We are continually working on improving UvA-BiTS services, including data processing and Virtual Lab functionality. In some cases, this has enabled us to extract more information from old GPS log files which is then incorporated into the database. These changes may result in slight differences when you reprocess historic data.

::* [[ DB_Model_2014 | e-Ecology data model]]
::* [[ UvAGPS|UvA-BiTS Data Model]] - OLD
::* [[Connecting to the UvA-BiTS database]]
::* [[Useful Queries]]
::* [[Queries for linking bird tracks to wind and flow data]]

==UvA-BiTS Services==

* Visualization Tools
:: [[KMZ Generator2]]: 
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth.
:: [[Calendar]]: 
:::Calendar overview with daily statistics of GPS-tracker
:: [[GPX]]: 
:::This tool produces GPX files from UvA-BiTS tracking data for dynamic [http://www.doarama.com Doarama] visualizations.
:: [[BirdView]]: 
:::Access meterological and radar data and overlay with GPS bird tracking data.
:: [[BirdSim]]: 
:::More dynamic version of BirdView that allows the user to scroll through weather data in time and space.
:: [[KMZ Generator|UvA-BiTS KMZ Generator]]: 
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth.

* [[Dashboard]]: These web pages allow UvA-BiTS users to get an overview of the raw data files of GPS bird tracking devices.

* [[ Annotation tables ]] for uploading GPS annotation data into your own schema. This service is currently not available.

== Changes in the Database in 2014 ==
'''Important'''
In the summer of 2014 we have '''re-arranged the administrative''' part of UvA-BiTS (and e-Ecology) in the database. End users may have to adapt their programs the following way:
:* The '''uva_*_limited''' views that used to exist in the gps schema are gone. Most of them should have been replaced with an equivalent '''ee_*_limited counterpart'''.
:* The new ee_*_limited views show a superset of the same information as before, but enhanced with new project information.
:* Some '''column names''' in the ee_*_limited views may differ from the previous views.
:* There are new limited views:
:** gps.ee_'''project'''_limited: shows your projects' information
:** gps.ee_'''shared'''_*_limited: set of data you are able to see related to trackers that a project is sharing with a project you belong to
:* Special attention: gps.uva_device_limited is now replaced by '''gps.ee_tracker_limited'''. For each tracker, you now see which project provides you access to that tracker.
:* Also, '''gps.uva_tracking_limited''' is gone. Instead of that we have gps.ee_tracker_speed_limited, which adds speed columns to the original data. This amounts to a merger of gps.ee_tracking_limited, gps.ee_tracking_speed_3d_limited and gps.ee_tracking_speed_limited.

==Other e-Ecology Datasets==

'''ENVIRONMENTAL DATA'''

::* [[ ECMWF|Weather Forecast Data (ECMWF)]]
::* [[ Elevation|Elevation Data ]]

'''FLYSAFE'''

:Bird data created from weather radar data and bird migration models. Financed by the Royal Netherlands Air Force.

::* [[Connecting to the Flysafe database]]
::* [[GPS|Flysafe GPS Data Model]]
::* [[ RADAR|Radar Data Model]]
::* [[ Wind profile|Wind Profile Data Model]]
::* [[ Fino1|Fino1 Data Model]]

==Other e-Ecology Services==
[[GPS tracks|FlySafe GPS Bird Track Generator]]: Produce kmz files of ARGOS/GPS bird tracking data from the FlySafe project to be viewed in Google Earth.

[[ Administration]]: Information about user rights and data processing tasks.

[[ Bookkeeping]]: Description of tables related to FlySafe data processing and archiving.

[[ ToDo ]]

E-Ecology Documentation and Guidance

2014-12-18T19:32:02Z

Judy: /* UvA-BiTS Services */

The e-Ecology Virtual Laboratories represent a collaborative effort between the University of Amsterdam Institute for Biodiversity and Ecosystem Dynamics and SARA Computing to manage bird tracking data and develop tools for data exploration, analysis and visualization. This wiki is designed to facilitate access to the data and tools within the Virtual Laboratories and to provide a forum for collaboration.

== Changes in the Database in 2014 ==
'''Important'''
In the summer of 2014 we have '''re-arranged the administrative''' part of UvA-BiTS (and e-Ecology) in the database. End users may have to adapt their programs the following way:
:* The '''uva_*_limited''' views that used to exist in the gps schema are gone. Most of them should have been replaced with an equivalent '''ee_*_limited counterpart'''.
:* The new ee_*_limited views show a superset of the same information as before, but enhanced with new project information.
:* Some '''column names''' in the ee_*_limited views may differ from the previous views.
:* There are new limited views:
:** gps.ee_'''project'''_limited: shows your projects' information
:** gps.ee_'''shared'''_*_limited: set of data you are able to see related to trackers that a project is sharing with a project you belong to
:* Special attention: gps.uva_device_limited is now replaced by '''gps.ee_tracker_limited'''. For each tracker, you now see which project provides you access to that tracker.
:* Also, '''gps.uva_tracking_limited''' is gone. Instead of that we have gps.ee_tracker_speed_limited, which adds speed columns to the original data. This amounts to a merger of gps.ee_tracking_limited, gps.ee_tracking_speed_3d_limited and gps.ee_tracking_speed_limited.

== UvA-BiTS data ==

:Bird data collected with the UvA-BiTS Bird Tracking System

Note: We are continually working on improving UvA-BiTS services, including data processing and Virtual Lab functionality. In some cases, this has enabled us to extract more information from old GPS log files which is then incorporated into the database. These changes may result in slight differences when you reprocess historic data.

::* [[ DB_Model_2014 | e-Ecology data model]]
::* [[ UvAGPS|UvA-BiTS Data Model]] - OLD
::* [[Connecting to the UvA-BiTS database]]
::* [[Useful Queries]]
::* [[Queries for linking bird tracks to wind and flow data]]

==UvA-BiTS Services==

* Visualization Tools
:: [[KMZ Generator2]]: 
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth.
:: [[Calendar]]: 
:::Calendar overview with daily statistics of GPS-tracker
:: [[GPX]]: 
:::This tool produces GPX files from UvA-BiTS tracking data for dynamic [http://www.doarama.com Doarama] visualizations.
:: [[BirdView]]: 
:::Access meterological and radar data and overlay with GPS bird tracking data.
:: [[BirdSim]]: 
:::More dynamic version of BirdView that allows the user to scroll through weather data in time and space.
:: [[KMZ Generator|UvA-BiTS KMZ Generator]]: 
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth.

* [[Dashboard]]: These web pages allow UvA-BiTS users to get an overview of the raw data files of GPS bird tracking devices.

* [[ Annotation tables ]] for uploading GPS annotation data into your own schema. This service is currently not available.

== Changes in the Database in 2014 ==
'''Important'''
In the summer of 2014 we have '''re-arranged the administrative''' part of UvA-BiTS (and e-Ecology) in the database. End users may have to adapt their programs the following way:
:* The '''uva_*_limited''' views that used to exist in the gps schema are gone. Most of them should have been replaced with an equivalent '''ee_*_limited counterpart'''.
:* The new ee_*_limited views show a superset of the same information as before, but enhanced with new project information.
:* Some '''column names''' in the ee_*_limited views may differ from the previous views.
:* There are new limited views:
:** gps.ee_'''project'''_limited: shows your projects' information
:** gps.ee_'''shared'''_*_limited: set of data you are able to see related to trackers that a project is sharing with a project you belong to
:* Special attention: gps.uva_device_limited is now replaced by '''gps.ee_tracker_limited'''. For each tracker, you now see which project provides you access to that tracker.
:* Also, '''gps.uva_tracking_limited''' is gone. Instead of that we have gps.ee_tracker_speed_limited, which adds speed columns to the original data. This amounts to a merger of gps.ee_tracking_limited, gps.ee_tracking_speed_3d_limited and gps.ee_tracking_speed_limited.

==Other e-Ecology Datasets==

'''ENVIRONMENTAL DATA'''

::* [[ ECMWF|Weather Forecast Data (ECMWF)]]
::* [[ Elevation|Elevation Data ]]

'''FLYSAFE'''

:Bird data created from weather radar data and bird migration models. Financed by the Royal Netherlands Air Force.

::* [[Connecting to the Flysafe database]]
::* [[GPS|Flysafe GPS Data Model]]
::* [[ RADAR|Radar Data Model]]
::* [[ Wind profile|Wind Profile Data Model]]
::* [[ Fino1|Fino1 Data Model]]

==Other e-Ecology Services==
[[GPS tracks|FlySafe GPS Bird Track Generator]]: Produce kmz files of ARGOS/GPS bird tracking data from the FlySafe project to be viewed in Google Earth.

[[ Administration]]: Information about user rights and data processing tasks.

[[ Bookkeeping]]: Description of tables related to FlySafe data processing and archiving.

[[ ToDo ]]

==Other e-Ecology Datasets==

'''ENVIRONMENTAL DATA'''

::* [[ ECMWF|Weather Forecast Data (ECMWF)]]
::* [[ Elevation|Elevation Data ]]

'''FLYSAFE'''

:Bird data created from weather radar data and bird migration models. Financed by the Royal Netherlands Air Force.

::* [[Connecting to the Flysafe database]]
::* [[GPS|Flysafe GPS Data Model]]
::* [[ RADAR|Radar Data Model]]
::* [[ Wind profile|Wind Profile Data Model]]
::* [[ Fino1|Fino1 Data Model]]

E-Ecology Documentation and Guidance

2014-12-18T19:28:37Z

Judy: /* UvA-BiTS Services */

The e-Ecology Virtual Laboratories represent a collaborative effort between the University of Amsterdam Institute for Biodiversity and Ecosystem Dynamics and SARA Computing to manage bird tracking data and develop tools for data exploration, analysis and visualization. This wiki is designed to facilitate access to the data and tools within the Virtual Laboratories and to provide a forum for collaboration.

== Changes in the Database in 2014 ==
'''Important'''
In the summer of 2014 we have '''re-arranged the administrative''' part of UvA-BiTS (and e-Ecology) in the database. End users may have to adapt their programs the following way:
:* The '''uva_*_limited''' views that used to exist in the gps schema are gone. Most of them should have been replaced with an equivalent '''ee_*_limited counterpart'''.
:* The new ee_*_limited views show a superset of the same information as before, but enhanced with new project information.
:* Some '''column names''' in the ee_*_limited views may differ from the previous views.
:* There are new limited views:
:** gps.ee_'''project'''_limited: shows your projects' information
:** gps.ee_'''shared'''_*_limited: set of data you are able to see related to trackers that a project is sharing with a project you belong to
:* Special attention: gps.uva_device_limited is now replaced by '''gps.ee_tracker_limited'''. For each tracker, you now see which project provides you access to that tracker.
:* Also, '''gps.uva_tracking_limited''' is gone. Instead of that we have gps.ee_tracker_speed_limited, which adds speed columns to the original data. This amounts to a merger of gps.ee_tracking_limited, gps.ee_tracking_speed_3d_limited and gps.ee_tracking_speed_limited.

== UvA-BiTS data ==

:Bird data collected with the UvA-BiTS Bird Tracking System

Note: We are continually working on improving UvA-BiTS services, including data processing and Virtual Lab functionality. In some cases, this has enabled us to extract more information from old GPS log files which is then incorporated into the database. These changes may result in slight differences when you reprocess historic data.

::* [[ DB_Model_2014 | e-Ecology data model]]
::* [[ UvAGPS|UvA-BiTS Data Model]] - OLD
::* [[Connecting to the UvA-BiTS database]]
::* [[Useful Queries]]
::* [[Queries for linking bird tracks to wind and flow data]]

==UvA-BiTS Services==

* Visualization Tools
:: [[KMZ Generator2]]: 
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth.
:: [[Calendar]]: 
:::Calendar overview with daily statistics of GPS-tracker
:: [[GPX]]: 
:::GPX file generator for [http://www.doarama.com Doarama] visualization.
:: [[BirdView]]: 
:::Access meterological and radar data and overlay with GPS bird tracking data.
:: [[BirdSim]]: 
:::More dynamic version of BirdView that allows the user to scroll through weather data in time and space.
:: [[KMZ Generator|UvA-BiTS KMZ Generator]]: 
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth.

* [[Dashboard]]: These web pages allow UvA-BiTS users to get an overview of the raw data files of GPS bird tracking devices.

* [[ Annotation tables ]] for uploading GPS annotation data into your own schema. This service is currently not available.

== Changes in the Database in 2014 ==
'''Important'''
In the summer of 2014 we have '''re-arranged the administrative''' part of UvA-BiTS (and e-Ecology) in the database. End users may have to adapt their programs the following way:
:* The '''uva_*_limited''' views that used to exist in the gps schema are gone. Most of them should have been replaced with an equivalent '''ee_*_limited counterpart'''.
:* The new ee_*_limited views show a superset of the same information as before, but enhanced with new project information.
:* Some '''column names''' in the ee_*_limited views may differ from the previous views.
:* There are new limited views:
:** gps.ee_'''project'''_limited: shows your projects' information
:** gps.ee_'''shared'''_*_limited: set of data you are able to see related to trackers that a project is sharing with a project you belong to
:* Special attention: gps.uva_device_limited is now replaced by '''gps.ee_tracker_limited'''. For each tracker, you now see which project provides you access to that tracker.
:* Also, '''gps.uva_tracking_limited''' is gone. Instead of that we have gps.ee_tracker_speed_limited, which adds speed columns to the original data. This amounts to a merger of gps.ee_tracking_limited, gps.ee_tracking_speed_3d_limited and gps.ee_tracking_speed_limited.

==Other e-Ecology Datasets==

'''ENVIRONMENTAL DATA'''

::* [[ ECMWF|Weather Forecast Data (ECMWF)]]
::* [[ Elevation|Elevation Data ]]

'''FLYSAFE'''

:Bird data created from weather radar data and bird migration models. Financed by the Royal Netherlands Air Force.

::* [[Connecting to the Flysafe database]]
::* [[GPS|Flysafe GPS Data Model]]
::* [[ RADAR|Radar Data Model]]
::* [[ Wind profile|Wind Profile Data Model]]
::* [[ Fino1|Fino1 Data Model]]

==Other e-Ecology Services==
[[GPS tracks|FlySafe GPS Bird Track Generator]]: Produce kmz files of ARGOS/GPS bird tracking data from the FlySafe project to be viewed in Google Earth.

[[ Administration]]: Information about user rights and data processing tasks.

[[ Bookkeeping]]: Description of tables related to FlySafe data processing and archiving.

[[ ToDo ]]

==Other e-Ecology Datasets==

'''ENVIRONMENTAL DATA'''

::* [[ ECMWF|Weather Forecast Data (ECMWF)]]
::* [[ Elevation|Elevation Data ]]

'''FLYSAFE'''

:Bird data created from weather radar data and bird migration models. Financed by the Royal Netherlands Air Force.

::* [[Connecting to the Flysafe database]]
::* [[GPS|Flysafe GPS Data Model]]
::* [[ RADAR|Radar Data Model]]
::* [[ Wind profile|Wind Profile Data Model]]
::* [[ Fino1|Fino1 Data Model]]

E-Ecology Documentation and Guidance

2014-12-18T19:25:14Z

Judy: /* UvA-BiTS Services */

The e-Ecology Virtual Laboratories represent a collaborative effort between the University of Amsterdam Institute for Biodiversity and Ecosystem Dynamics and SARA Computing to manage bird tracking data and develop tools for data exploration, analysis and visualization. This wiki is designed to facilitate access to the data and tools within the Virtual Laboratories and to provide a forum for collaboration.

== Changes in the Database in 2014 ==
'''Important'''
In the summer of 2014 we have '''re-arranged the administrative''' part of UvA-BiTS (and e-Ecology) in the database. End users may have to adapt their programs the following way:
:* The '''uva_*_limited''' views that used to exist in the gps schema are gone. Most of them should have been replaced with an equivalent '''ee_*_limited counterpart'''.
:* The new ee_*_limited views show a superset of the same information as before, but enhanced with new project information.
:* Some '''column names''' in the ee_*_limited views may differ from the previous views.
:* There are new limited views:
:** gps.ee_'''project'''_limited: shows your projects' information
:** gps.ee_'''shared'''_*_limited: set of data you are able to see related to trackers that a project is sharing with a project you belong to
:* Special attention: gps.uva_device_limited is now replaced by '''gps.ee_tracker_limited'''. For each tracker, you now see which project provides you access to that tracker.
:* Also, '''gps.uva_tracking_limited''' is gone. Instead of that we have gps.ee_tracker_speed_limited, which adds speed columns to the original data. This amounts to a merger of gps.ee_tracking_limited, gps.ee_tracking_speed_3d_limited and gps.ee_tracking_speed_limited.

== UvA-BiTS data ==

:Bird data collected with the UvA-BiTS Bird Tracking System

Note: We are continually working on improving UvA-BiTS services, including data processing and Virtual Lab functionality. In some cases, this has enabled us to extract more information from old GPS log files which is then incorporated into the database. These changes may result in slight differences when you reprocess historic data.

::* [[ DB_Model_2014 | e-Ecology data model]]
::* [[ UvAGPS|UvA-BiTS Data Model]] - OLD
::* [[Connecting to the UvA-BiTS database]]
::* [[Useful Queries]]
::* [[Queries for linking bird tracks to wind and flow data]]

==UvA-BiTS Services==

* Visualization Tools
:: [[KMZ Generator|UvA-BiTS KMZ Generator]]: 
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth.
:: [[BirdView]]: 
:::Access meterological and radar data and overlay with GPS bird tracking data.
:: [[BirdSim]]: 
:::More dynamic version of BirdView that allows the user to scroll through weather data in time and space.
:: [[KMZ Generator2]]: 
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth.
:: [[Calendar]]: 
:::Calendar overview with daily statistics of GPS-tracker
:: [[GPX]]: 
:::GPX file generator for [http://www.doarama.com Doarama] visualization.

* [[Dashboard]]: These web pages allow UvA-BiTS users to get an overview of the raw data files of GPS bird tracking devices.

* [[ Annotation tables ]] for uploading GPS annotation data into your own schema. This service is currently not available.

== Changes in the Database in 2014 ==
'''Important'''
In the summer of 2014 we have '''re-arranged the administrative''' part of UvA-BiTS (and e-Ecology) in the database. End users may have to adapt their programs the following way:
:* The '''uva_*_limited''' views that used to exist in the gps schema are gone. Most of them should have been replaced with an equivalent '''ee_*_limited counterpart'''.
:* The new ee_*_limited views show a superset of the same information as before, but enhanced with new project information.
:* Some '''column names''' in the ee_*_limited views may differ from the previous views.
:* There are new limited views:
:** gps.ee_'''project'''_limited: shows your projects' information
:** gps.ee_'''shared'''_*_limited: set of data you are able to see related to trackers that a project is sharing with a project you belong to
:* Special attention: gps.uva_device_limited is now replaced by '''gps.ee_tracker_limited'''. For each tracker, you now see which project provides you access to that tracker.
:* Also, '''gps.uva_tracking_limited''' is gone. Instead of that we have gps.ee_tracker_speed_limited, which adds speed columns to the original data. This amounts to a merger of gps.ee_tracking_limited, gps.ee_tracking_speed_3d_limited and gps.ee_tracking_speed_limited.

==Other e-Ecology Datasets==

'''ENVIRONMENTAL DATA'''

::* [[ ECMWF|Weather Forecast Data (ECMWF)]]
::* [[ Elevation|Elevation Data ]]

'''FLYSAFE'''

:Bird data created from weather radar data and bird migration models. Financed by the Royal Netherlands Air Force.

::* [[Connecting to the Flysafe database]]
::* [[GPS|Flysafe GPS Data Model]]
::* [[ RADAR|Radar Data Model]]
::* [[ Wind profile|Wind Profile Data Model]]
::* [[ Fino1|Fino1 Data Model]]

==Other e-Ecology Services==
[[GPS tracks|FlySafe GPS Bird Track Generator]]: Produce kmz files of ARGOS/GPS bird tracking data from the FlySafe project to be viewed in Google Earth.

[[ Administration]]: Information about user rights and data processing tasks.

[[ Bookkeeping]]: Description of tables related to FlySafe data processing and archiving.

[[ ToDo ]]

==Other e-Ecology Datasets==

'''ENVIRONMENTAL DATA'''

::* [[ ECMWF|Weather Forecast Data (ECMWF)]]
::* [[ Elevation|Elevation Data ]]

'''FLYSAFE'''

:Bird data created from weather radar data and bird migration models. Financed by the Royal Netherlands Air Force.

::* [[Connecting to the Flysafe database]]
::* [[GPS|Flysafe GPS Data Model]]
::* [[ RADAR|Radar Data Model]]
::* [[ Wind profile|Wind Profile Data Model]]
::* [[ Fino1|Fino1 Data Model]]

E-Ecology Documentation and Guidance

2014-12-18T19:22:34Z

Judy: /* Other e-Ecology Services */

The e-Ecology Virtual Laboratories represent a collaborative effort between the University of Amsterdam Institute for Biodiversity and Ecosystem Dynamics and SARA Computing to manage bird tracking data and develop tools for data exploration, analysis and visualization. This wiki is designed to facilitate access to the data and tools within the Virtual Laboratories and to provide a forum for collaboration.

== Changes in the Database in 2014 ==
'''Important'''
In the summer of 2014 we have '''re-arranged the administrative''' part of UvA-BiTS (and e-Ecology) in the database. End users may have to adapt their programs the following way:
:* The '''uva_*_limited''' views that used to exist in the gps schema are gone. Most of them should have been replaced with an equivalent '''ee_*_limited counterpart'''.
:* The new ee_*_limited views show a superset of the same information as before, but enhanced with new project information.
:* Some '''column names''' in the ee_*_limited views may differ from the previous views.
:* There are new limited views:
:** gps.ee_'''project'''_limited: shows your projects' information
:** gps.ee_'''shared'''_*_limited: set of data you are able to see related to trackers that a project is sharing with a project you belong to
:* Special attention: gps.uva_device_limited is now replaced by '''gps.ee_tracker_limited'''. For each tracker, you now see which project provides you access to that tracker.
:* Also, '''gps.uva_tracking_limited''' is gone. Instead of that we have gps.ee_tracker_speed_limited, which adds speed columns to the original data. This amounts to a merger of gps.ee_tracking_limited, gps.ee_tracking_speed_3d_limited and gps.ee_tracking_speed_limited.

== UvA-BiTS data ==

:Bird data collected with the UvA-BiTS Bird Tracking System

Note: We are continually working on improving UvA-BiTS services, including data processing and Virtual Lab functionality. In some cases, this has enabled us to extract more information from old GPS log files which is then incorporated into the database. These changes may result in slight differences when you reprocess historic data.

::* [[ DB_Model_2014 | e-Ecology data model]]
::* [[ UvAGPS|UvA-BiTS Data Model]] - OLD
::* [[Connecting to the UvA-BiTS database]]
::* [[Useful Queries]]
::* [[Queries for linking bird tracks to wind and flow data]]

==UvA-BiTS Services==

* Visualization Tools
:: [[KMZ Generator|UvA-BiTS KMZ Generator]]: 
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth.
:: [[BirdView]]: 
:::Access meterological and radar data and overlay with GPS bird tracking data.
:: [[BirdSim]]: 
:::More dynamic version of BirdView that allows the user to scroll through weather data in time and space.
:: [[KMZ Generator2]]: 
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth.
:: [[Calendar]]: 
:::Calendar overview with daily statistics of GPS-tracker
:: [[GPX]]: 
:::GPX file generator for [http://www.doarama.com Doarama] visualization.

* [[ Annotation tables ]] for uploading GPS annotation data into your own schema

* [[Dashboard]]: These web pages allow UvA-BiTS users to get an overview of the raw data files of GPS bird tracking devices.

== Changes in the Database in 2014 ==
'''Important'''
In the summer of 2014 we have '''re-arranged the administrative''' part of UvA-BiTS (and e-Ecology) in the database. End users may have to adapt their programs the following way:
:* The '''uva_*_limited''' views that used to exist in the gps schema are gone. Most of them should have been replaced with an equivalent '''ee_*_limited counterpart'''.
:* The new ee_*_limited views show a superset of the same information as before, but enhanced with new project information.
:* Some '''column names''' in the ee_*_limited views may differ from the previous views.
:* There are new limited views:
:** gps.ee_'''project'''_limited: shows your projects' information
:** gps.ee_'''shared'''_*_limited: set of data you are able to see related to trackers that a project is sharing with a project you belong to
:* Special attention: gps.uva_device_limited is now replaced by '''gps.ee_tracker_limited'''. For each tracker, you now see which project provides you access to that tracker.
:* Also, '''gps.uva_tracking_limited''' is gone. Instead of that we have gps.ee_tracker_speed_limited, which adds speed columns to the original data. This amounts to a merger of gps.ee_tracking_limited, gps.ee_tracking_speed_3d_limited and gps.ee_tracking_speed_limited.

==Other e-Ecology Datasets==

'''ENVIRONMENTAL DATA'''

::* [[ ECMWF|Weather Forecast Data (ECMWF)]]
::* [[ Elevation|Elevation Data ]]

'''FLYSAFE'''

:Bird data created from weather radar data and bird migration models. Financed by the Royal Netherlands Air Force.

::* [[Connecting to the Flysafe database]]
::* [[GPS|Flysafe GPS Data Model]]
::* [[ RADAR|Radar Data Model]]
::* [[ Wind profile|Wind Profile Data Model]]
::* [[ Fino1|Fino1 Data Model]]

==Other e-Ecology Services==
[[GPS tracks|FlySafe GPS Bird Track Generator]]: Produce kmz files of ARGOS/GPS bird tracking data from the FlySafe project to be viewed in Google Earth.

[[ Administration]]: Information about user rights and data processing tasks.

[[ Bookkeeping]]: Description of tables related to FlySafe data processing and archiving.

[[ ToDo ]]

==Other e-Ecology Datasets==

'''ENVIRONMENTAL DATA'''

::* [[ ECMWF|Weather Forecast Data (ECMWF)]]
::* [[ Elevation|Elevation Data ]]

'''FLYSAFE'''

:Bird data created from weather radar data and bird migration models. Financed by the Royal Netherlands Air Force.

::* [[Connecting to the Flysafe database]]
::* [[GPS|Flysafe GPS Data Model]]
::* [[ RADAR|Radar Data Model]]
::* [[ Wind profile|Wind Profile Data Model]]
::* [[ Fino1|Fino1 Data Model]]

E-Ecology Documentation and Guidance

2014-12-18T19:21:39Z

Judy: /* Other e-Ecology Services */

The e-Ecology Virtual Laboratories represent a collaborative effort between the University of Amsterdam Institute for Biodiversity and Ecosystem Dynamics and SARA Computing to manage bird tracking data and develop tools for data exploration, analysis and visualization. This wiki is designed to facilitate access to the data and tools within the Virtual Laboratories and to provide a forum for collaboration.

== Changes in the Database in 2014 ==
'''Important'''
In the summer of 2014 we have '''re-arranged the administrative''' part of UvA-BiTS (and e-Ecology) in the database. End users may have to adapt their programs the following way:
:* The '''uva_*_limited''' views that used to exist in the gps schema are gone. Most of them should have been replaced with an equivalent '''ee_*_limited counterpart'''.
:* The new ee_*_limited views show a superset of the same information as before, but enhanced with new project information.
:* Some '''column names''' in the ee_*_limited views may differ from the previous views.
:* There are new limited views:
:** gps.ee_'''project'''_limited: shows your projects' information
:** gps.ee_'''shared'''_*_limited: set of data you are able to see related to trackers that a project is sharing with a project you belong to
:* Special attention: gps.uva_device_limited is now replaced by '''gps.ee_tracker_limited'''. For each tracker, you now see which project provides you access to that tracker.
:* Also, '''gps.uva_tracking_limited''' is gone. Instead of that we have gps.ee_tracker_speed_limited, which adds speed columns to the original data. This amounts to a merger of gps.ee_tracking_limited, gps.ee_tracking_speed_3d_limited and gps.ee_tracking_speed_limited.

== UvA-BiTS data ==

:Bird data collected with the UvA-BiTS Bird Tracking System

Note: We are continually working on improving UvA-BiTS services, including data processing and Virtual Lab functionality. In some cases, this has enabled us to extract more information from old GPS log files which is then incorporated into the database. These changes may result in slight differences when you reprocess historic data.

::* [[ DB_Model_2014 | e-Ecology data model]]
::* [[ UvAGPS|UvA-BiTS Data Model]] - OLD
::* [[Connecting to the UvA-BiTS database]]
::* [[Useful Queries]]
::* [[Queries for linking bird tracks to wind and flow data]]

==UvA-BiTS Services==

* Visualization Tools
:: [[KMZ Generator|UvA-BiTS KMZ Generator]]: 
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth.
:: [[BirdView]]: 
:::Access meterological and radar data and overlay with GPS bird tracking data.
:: [[BirdSim]]: 
:::More dynamic version of BirdView that allows the user to scroll through weather data in time and space.
:: [[KMZ Generator2]]: 
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth.
:: [[Calendar]]: 
:::Calendar overview with daily statistics of GPS-tracker
:: [[GPX]]: 
:::GPX file generator for [http://www.doarama.com Doarama] visualization.

* [[ Annotation tables ]] for uploading GPS annotation data into your own schema

* [[Dashboard]]: These web pages allow UvA-BiTS users to get an overview of the raw data files of GPS bird tracking devices.

== Changes in the Database in 2014 ==
'''Important'''
In the summer of 2014 we have '''re-arranged the administrative''' part of UvA-BiTS (and e-Ecology) in the database. End users may have to adapt their programs the following way:
:* The '''uva_*_limited''' views that used to exist in the gps schema are gone. Most of them should have been replaced with an equivalent '''ee_*_limited counterpart'''.
:* The new ee_*_limited views show a superset of the same information as before, but enhanced with new project information.
:* Some '''column names''' in the ee_*_limited views may differ from the previous views.
:* There are new limited views:
:** gps.ee_'''project'''_limited: shows your projects' information
:** gps.ee_'''shared'''_*_limited: set of data you are able to see related to trackers that a project is sharing with a project you belong to
:* Special attention: gps.uva_device_limited is now replaced by '''gps.ee_tracker_limited'''. For each tracker, you now see which project provides you access to that tracker.
:* Also, '''gps.uva_tracking_limited''' is gone. Instead of that we have gps.ee_tracker_speed_limited, which adds speed columns to the original data. This amounts to a merger of gps.ee_tracking_limited, gps.ee_tracking_speed_3d_limited and gps.ee_tracking_speed_limited.

==Other e-Ecology Services==
[[GPS tracks|FlySafe GPS Bird Track Generator]]: Produce kmz files of ARGOS/GPS bird tracking data from the FlySafe project to be viewed in Google Earth.

[[ Administration]]: Information about user rights and data processing tasks.

[[ Bookkeeping]]: Description of tables related to FlySafe data processing and archiving.

[[ ToDo ]]

==Other e-Ecology Datasets==

'''ENVIRONMENTAL DATA'''

::* [[ ECMWF|Weather Forecast Data (ECMWF)]]
::* [[ Elevation|Elevation Data ]]

'''FLYSAFE'''

:Bird data created from weather radar data and bird migration models. Financed by the Royal Netherlands Air Force.

::* [[Connecting to the Flysafe database]]
::* [[GPS|Flysafe GPS Data Model]]
::* [[ RADAR|Radar Data Model]]
::* [[ Wind profile|Wind Profile Data Model]]
::* [[ Fino1|Fino1 Data Model]]

E-Ecology Documentation and Guidance

2014-12-18T19:19:43Z

Judy: /* UvA-BiTS Services */

The e-Ecology Virtual Laboratories represent a collaborative effort between the University of Amsterdam Institute for Biodiversity and Ecosystem Dynamics and SARA Computing to manage bird tracking data and develop tools for data exploration, analysis and visualization. This wiki is designed to facilitate access to the data and tools within the Virtual Laboratories and to provide a forum for collaboration.

== Changes in the Database in 2014 ==
'''Important'''
In the summer of 2014 we have '''re-arranged the administrative''' part of UvA-BiTS (and e-Ecology) in the database. End users may have to adapt their programs the following way:
:* The '''uva_*_limited''' views that used to exist in the gps schema are gone. Most of them should have been replaced with an equivalent '''ee_*_limited counterpart'''.
:* The new ee_*_limited views show a superset of the same information as before, but enhanced with new project information.
:* Some '''column names''' in the ee_*_limited views may differ from the previous views.
:* There are new limited views:
:** gps.ee_'''project'''_limited: shows your projects' information
:** gps.ee_'''shared'''_*_limited: set of data you are able to see related to trackers that a project is sharing with a project you belong to
:* Special attention: gps.uva_device_limited is now replaced by '''gps.ee_tracker_limited'''. For each tracker, you now see which project provides you access to that tracker.
:* Also, '''gps.uva_tracking_limited''' is gone. Instead of that we have gps.ee_tracker_speed_limited, which adds speed columns to the original data. This amounts to a merger of gps.ee_tracking_limited, gps.ee_tracking_speed_3d_limited and gps.ee_tracking_speed_limited.

== UvA-BiTS data ==

:Bird data collected with the UvA-BiTS Bird Tracking System

Note: We are continually working on improving UvA-BiTS services, including data processing and Virtual Lab functionality. In some cases, this has enabled us to extract more information from old GPS log files which is then incorporated into the database. These changes may result in slight differences when you reprocess historic data.

::* [[ DB_Model_2014 | e-Ecology data model]]
::* [[ UvAGPS|UvA-BiTS Data Model]] - OLD
::* [[Connecting to the UvA-BiTS database]]
::* [[Useful Queries]]
::* [[Queries for linking bird tracks to wind and flow data]]

==UvA-BiTS Services==

* Visualization Tools
:: [[KMZ Generator|UvA-BiTS KMZ Generator]]: 
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth.
:: [[BirdView]]: 
:::Access meterological and radar data and overlay with GPS bird tracking data.
:: [[BirdSim]]: 
:::More dynamic version of BirdView that allows the user to scroll through weather data in time and space.
:: [[KMZ Generator2]]: 
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth.
:: [[Calendar]]: 
:::Calendar overview with daily statistics of GPS-tracker
:: [[GPX]]: 
:::GPX file generator for [http://www.doarama.com Doarama] visualization.

* [[ Annotation tables ]] for uploading GPS annotation data into your own schema

* [[Dashboard]]: These web pages allow UvA-BiTS users to get an overview of the raw data files of GPS bird tracking devices.

== Changes in the Database in 2014 ==
'''Important'''
In the summer of 2014 we have '''re-arranged the administrative''' part of UvA-BiTS (and e-Ecology) in the database. End users may have to adapt their programs the following way:
:* The '''uva_*_limited''' views that used to exist in the gps schema are gone. Most of them should have been replaced with an equivalent '''ee_*_limited counterpart'''.
:* The new ee_*_limited views show a superset of the same information as before, but enhanced with new project information.
:* Some '''column names''' in the ee_*_limited views may differ from the previous views.
:* There are new limited views:
:** gps.ee_'''project'''_limited: shows your projects' information
:** gps.ee_'''shared'''_*_limited: set of data you are able to see related to trackers that a project is sharing with a project you belong to
:* Special attention: gps.uva_device_limited is now replaced by '''gps.ee_tracker_limited'''. For each tracker, you now see which project provides you access to that tracker.
:* Also, '''gps.uva_tracking_limited''' is gone. Instead of that we have gps.ee_tracker_speed_limited, which adds speed columns to the original data. This amounts to a merger of gps.ee_tracking_limited, gps.ee_tracking_speed_3d_limited and gps.ee_tracking_speed_limited.

==Other e-Ecology Services==
[[GPS tracks|FlySafe GPS Bird Track Generator]]: 
:Produce kmz files of ARGOS/GPS bird tracking data from the FlySafe project to be viewed in Google Earth.
[[ Administration]]: Information about user rights and data processing tasks.

==Other e-Ecology Datasets==

'''ENVIRONMENTAL DATA'''

::* [[ ECMWF|Weather Forecast Data (ECMWF)]]
::* [[ Elevation|Elevation Data ]]

'''FLYSAFE'''

:Bird data created from weather radar data and bird migration models. Financed by the Royal Netherlands Air Force.

::* [[Connecting to the Flysafe database]]
::* [[GPS|Flysafe GPS Data Model]]
::* [[ RADAR|Radar Data Model]]
::* [[ Wind profile|Wind Profile Data Model]]
::* [[ Fino1|Fino1 Data Model]]

E-Ecology Documentation and Guidance

2014-12-18T14:46:14Z

Judy: /* UvA-BiTS Services */

The e-Ecology Virtual Laboratories represent a collaborative effort between the University of Amsterdam Institute for Biodiversity and Ecosystem Dynamics and SARA Computing to manage bird tracking data and develop tools for data exploration, analysis and visualization. This wiki is designed to facilitate access to the data and tools within the Virtual Laboratories and to provide a forum for collaboration.

== Changes in the Database in 2014 ==
'''Important'''
In the summer of 2014 we have '''re-arranged the administrative''' part of UvA-BiTS (and e-Ecology) in the database. End users may have to adapt their programs the following way:
:* The '''uva_*_limited''' views that used to exist in the gps schema are gone. Most of them should have been replaced with an equivalent '''ee_*_limited counterpart'''.
:* The new ee_*_limited views show a superset of the same information as before, but enhanced with new project information.
:* Some '''column names''' in the ee_*_limited views may differ from the previous views.
:* There are new limited views:
:** gps.ee_'''project'''_limited: shows your projects' information
:** gps.ee_'''shared'''_*_limited: set of data you are able to see related to trackers that a project is sharing with a project you belong to
:* Special attention: gps.uva_device_limited is now replaced by '''gps.ee_tracker_limited'''. For each tracker, you now see which project provides you access to that tracker.
:* Also, '''gps.uva_tracking_limited''' is gone. Instead of that we have gps.ee_tracker_speed_limited, which adds speed columns to the original data. This amounts to a merger of gps.ee_tracking_limited, gps.ee_tracking_speed_3d_limited and gps.ee_tracking_speed_limited.

== UvA-BiTS data ==

:Bird data collected with the UvA-BiTS Bird Tracking System

Note: We are continually working on improving UvA-BiTS services, including data processing and Virtual Lab functionality. In some cases, this has enabled us to extract more information from old GPS log files which is then incorporated into the database. These changes may result in slight differences when you reprocess historic data.

::* [[ DB_Model_2014 | e-Ecology data model]]
::* [[ UvAGPS|UvA-BiTS Data Model]] - OLD
::* [[Connecting to the UvA-BiTS database]]
::* [[Useful Queries]]
::* [[Queries for linking bird tracks to wind and flow data]]

==UvA-BiTS Services==

* Visualization Tools
:: [[KMZ Generator|UvA-BiTS KMZ Generator]]: 
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth.
:: [[BirdView]]: 
:::Access meterological and radar data and overlay with GPS bird tracking data.
:: [[BirdSim]]: 
:::More dynamic version of BirdView that allows the user to scroll through weather data in time and space.
:: [[KMZ Generator2]]: 
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth.
:: [[Calendar]]: 
:::Calendar overview with daily statistics of GPS-tracker
:: [[GPX]]: 
:::GPX file generator for [http://www.doarama.com Doarama] visualization.

* [[ Annotation tables ]] for uploading GPS annotation data into your own schema

* [[Dashboard]]: These web pages allow UvA-BiTS users to get an overview of the raw data files of GPS bird tracking devices.

* [[ Bookkeeping]]: Description of tables related to FlySafe data processing and archiving.

[[ ToDo ]]

== Changes in the Database in 2014 ==
'''Important'''
In the summer of 2014 we have '''re-arranged the administrative''' part of UvA-BiTS (and e-Ecology) in the database. End users may have to adapt their programs the following way:
:* The '''uva_*_limited''' views that used to exist in the gps schema are gone. Most of them should have been replaced with an equivalent '''ee_*_limited counterpart'''.
:* The new ee_*_limited views show a superset of the same information as before, but enhanced with new project information.
:* Some '''column names''' in the ee_*_limited views may differ from the previous views.
:* There are new limited views:
:** gps.ee_'''project'''_limited: shows your projects' information
:** gps.ee_'''shared'''_*_limited: set of data you are able to see related to trackers that a project is sharing with a project you belong to
:* Special attention: gps.uva_device_limited is now replaced by '''gps.ee_tracker_limited'''. For each tracker, you now see which project provides you access to that tracker.
:* Also, '''gps.uva_tracking_limited''' is gone. Instead of that we have gps.ee_tracker_speed_limited, which adds speed columns to the original data. This amounts to a merger of gps.ee_tracking_limited, gps.ee_tracking_speed_3d_limited and gps.ee_tracking_speed_limited.

==Other e-Ecology Services==
[[GPS tracks|FlySafe GPS Bird Track Generator]]: 
:Produce kmz files of ARGOS/GPS bird tracking data from the FlySafe project to be viewed in Google Earth.
[[ Administration]]: Information about user rights and data processing tasks.

==Other e-Ecology Datasets==

'''ENVIRONMENTAL DATA'''

::* [[ ECMWF|Weather Forecast Data (ECMWF)]]
::* [[ Elevation|Elevation Data ]]

'''FLYSAFE'''

:Bird data created from weather radar data and bird migration models. Financed by the Royal Netherlands Air Force.

::* [[Connecting to the Flysafe database]]
::* [[GPS|Flysafe GPS Data Model]]
::* [[ RADAR|Radar Data Model]]
::* [[ Wind profile|Wind Profile Data Model]]
::* [[ Fino1|Fino1 Data Model]]

E-Ecology Documentation and Guidance

2014-12-18T14:40:48Z

Judy: /* UvA-BiTS Services */

The e-Ecology Virtual Laboratories represent a collaborative effort between the University of Amsterdam Institute for Biodiversity and Ecosystem Dynamics and SARA Computing to manage bird tracking data and develop tools for data exploration, analysis and visualization. This wiki is designed to facilitate access to the data and tools within the Virtual Laboratories and to provide a forum for collaboration.

== Changes in the Database in 2014 ==
'''Important'''
In the summer of 2014 we have '''re-arranged the administrative''' part of UvA-BiTS (and e-Ecology) in the database. End users may have to adapt their programs the following way:
:* The '''uva_*_limited''' views that used to exist in the gps schema are gone. Most of them should have been replaced with an equivalent '''ee_*_limited counterpart'''.
:* The new ee_*_limited views show a superset of the same information as before, but enhanced with new project information.
:* Some '''column names''' in the ee_*_limited views may differ from the previous views.
:* There are new limited views:
:** gps.ee_'''project'''_limited: shows your projects' information
:** gps.ee_'''shared'''_*_limited: set of data you are able to see related to trackers that a project is sharing with a project you belong to
:* Special attention: gps.uva_device_limited is now replaced by '''gps.ee_tracker_limited'''. For each tracker, you now see which project provides you access to that tracker.
:* Also, '''gps.uva_tracking_limited''' is gone. Instead of that we have gps.ee_tracker_speed_limited, which adds speed columns to the original data. This amounts to a merger of gps.ee_tracking_limited, gps.ee_tracking_speed_3d_limited and gps.ee_tracking_speed_limited.

== UvA-BiTS data ==

:Bird data collected with the UvA-BiTS Bird Tracking System

Note: We are continually working on improving UvA-BiTS services, including data processing and Virtual Lab functionality. In some cases, this has enabled us to extract more information from old GPS log files which is then incorporated into the database. These changes may result in slight differences when you reprocess historic data.

::* [[ DB_Model_2014 | e-Ecology data model]]
::* [[ UvAGPS|UvA-BiTS Data Model]] - OLD
::* [[Connecting to the UvA-BiTS database]]
::* [[Useful Queries]]
::* [[Queries for linking bird tracks to wind and flow data]]

==UvA-BiTS Services==

* Visualization Tools
:: [[KMZ Generator|UvA-BiTS KMZ Generator]]: 
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth.
:: [[BirdView]]: 
:::Access meterological and radar data and overlay with GPS bird tracking data.
:: [[BirdSim]]: 
:::More dynamic version of BirdView that allows the user to scroll through weather data in time and space.
:: [[KMZ Generator2]]: 
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth.
:: [[Calendar]]: 
:::Calendar overview with daily statistics of GPS-tracker
:: [[GPX]]: 
:::GPX file generator for [http://www.doarama.com Doarama] visualization.

* [[ Annotation tables ]] for uploading GPS annotation data into your own schema

* [[Dashboard]]: These web pages allow UvA-BiTS users to get an overview of the raw data files of GPS bird tracking devices.
:: [[ FilesDashboard | Tracker files dashboard ]]

* [[ Bookkeeping]]: Description of tables related to FlySafe data processing and archiving.

[[ ToDo ]]

== Changes in the Database in 2014 ==
'''Important'''
In the summer of 2014 we have '''re-arranged the administrative''' part of UvA-BiTS (and e-Ecology) in the database. End users may have to adapt their programs the following way:
:* The '''uva_*_limited''' views that used to exist in the gps schema are gone. Most of them should have been replaced with an equivalent '''ee_*_limited counterpart'''.
:* The new ee_*_limited views show a superset of the same information as before, but enhanced with new project information.
:* Some '''column names''' in the ee_*_limited views may differ from the previous views.
:* There are new limited views:
:** gps.ee_'''project'''_limited: shows your projects' information
:** gps.ee_'''shared'''_*_limited: set of data you are able to see related to trackers that a project is sharing with a project you belong to
:* Special attention: gps.uva_device_limited is now replaced by '''gps.ee_tracker_limited'''. For each tracker, you now see which project provides you access to that tracker.
:* Also, '''gps.uva_tracking_limited''' is gone. Instead of that we have gps.ee_tracker_speed_limited, which adds speed columns to the original data. This amounts to a merger of gps.ee_tracking_limited, gps.ee_tracking_speed_3d_limited and gps.ee_tracking_speed_limited.

==Other e-Ecology Services==
[[GPS tracks|FlySafe GPS Bird Track Generator]]: 
:Produce kmz files of ARGOS/GPS bird tracking data from the FlySafe project to be viewed in Google Earth.
[[ Administration]]: Information about user rights and data processing tasks.

==Other e-Ecology Datasets==

'''ENVIRONMENTAL DATA'''

::* [[ ECMWF|Weather Forecast Data (ECMWF)]]
::* [[ Elevation|Elevation Data ]]

'''FLYSAFE'''

:Bird data created from weather radar data and bird migration models. Financed by the Royal Netherlands Air Force.

::* [[Connecting to the Flysafe database]]
::* [[GPS|Flysafe GPS Data Model]]
::* [[ RADAR|Radar Data Model]]
::* [[ Wind profile|Wind Profile Data Model]]
::* [[ Fino1|Fino1 Data Model]]

E-Ecology Documentation and Guidance

2014-12-18T14:38:28Z

Judy: /* Other e-Ecology Services */

E-Ecology Documentation and Guidance

2014-12-18T14:37:31Z

Judy: /* Other e-Ecology Services */

E-Ecology Documentation and Guidance

2014-12-18T14:37:17Z

Judy: /* Other e-Ecology Services */

E-Ecology Documentation and Guidance

2014-12-18T14:37:04Z

Judy: /* UvA-BiTS Services */

E-Ecology Documentation and Guidance

2014-12-18T14:36:32Z

Judy: /* Other e-Ecology Services */

E-Ecology Documentation and Guidance

2014-12-18T14:36:22Z

Judy: /* UvA-BiTS Services */

E-Ecology Documentation and Guidance

2014-12-18T14:33:56Z

Judy: /* UvA-BiTS Services */

E-Ecology Documentation and Guidance

2014-12-18T13:30:55Z

Judy: /* Services */

E-Ecology Documentation and Guidance

2014-12-18T13:29:39Z

Judy: /* Datasets */

UvA-BiTS Tracking Data

2014-12-11T13:44:39Z

Judy: /* gps.ee_acc_start_limited */

== Tracking data ==

'''Note:''' These views will show no rows for trackers not yet attached to birds.

=== ''gps.''ee_tracking_speed_limited ===

Shows the tracking data that your projects allow you to see.

Each row represents a GPS fix as it is interpreted from the data that comes via the GPS tracker, plus it adds speeds.

It replaces old views gps.uva_tracking_limited, gps.uva_tracking_speed_3d_limited and gps.uva_tracking_speed. This table structure has ''not'' changed with respect to our previous database.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker that reported the fix. Part of the unique identifier of the GPS fix.
|-
|date_time||timestamp||NO||Time when the fix was taken. UTC (i.e.: no timezone). Part of the unique identifier of the GPS fix.
|-
|latitude||float8||YES||Latitude of the GPS fix, decimal degrees.
|-
|longitude||float8||YES||Longitude of the GPS fix, decimal degrees.
|-
|altitude||int4||YES||Altitude above sea level measured by GPS tag in meters
|-
|pressure||int4||YES||Pressure measured by GPS tag sensor in Pascals
|-
|temperature||float8||YES||Temperature measured by GPS tag sensor in Celsius
|-
|h_accuracy||float8||YES||Horizontal accuracy (meters)
|-
|v_accuracy||float8||YES||vertical accuracy (meters)
|-
|x_speed||float8||YES||meters/second
|-
|y_speed||float8||YES||meters/second
|-
|z_speed||float8||YES||meters/second
|-
|gps_fixtime||float8||YES||GPS fix time (also known as Time to Fix) in seconds
|-
|userflag||int4||YES||Acceptable = 0; Data flagged as unacceptable by user if not equal to 0.
|-
|satellites_used||int2||YES||Number of satellites used for fix the GPS measurement
|-
|positiondop||float8||YES||Positional DOP (Dilution of Precision), unitless
|-
|speed_accuracy||float8||YES||
|-
|location||geometry||YES||postgreSQL geometry; to allow GIS operations
|-
|vnorth||float8||YES||- vx * sin(lat) * cos(long) - vy * sin(lat) * sin (long) + vz * cos(lat), m/s
|-
|veast||float8||YES||- vx * sin(lon) + vy * cos(lon), m/s
|-
|vdown||float8||YES||- vx * cos(lat) * cos(lon) - vy * cos(lat) * sin(lon) - vz * sin(lat), m/s
|-
|speed||float8||YES||- vx * sin(lon) + vy * cos(lon)
|-
|speed_3d||float8||YES|| sqrt(d.x_speed^2 + d.y_speed^2 + d.z_speed^2), m/s
|-
|speed_2d||float8||YES|| Speed on 2D, m/s
|-
|direction||numeric||YES|| Heading in degrees; 0 is north, 90 is east
|-
|altitude_agl||float8||YES|| Reported GPS altitude MINUS terrain elevation on the coordinate based on Digital Elevation Model data used in the e-ecology database (for further information see: [[Elevation]]
|}

=== ''gps.''ee_acceleration_limited ===

Acceleration is measured in blocks of 1-200 measurements (records). The date_time mentioned in this block is the date and time of the first acceleration measurement. Sequential measurements within a block are numbered with an index 1... (max 200). Every row represents one measurement in 3 directions (X,Y,Z). The order is important within each block, so you should always sort the acceleration measurements by block and index. If you need the exact time for each row, you can calculate it from the index, the sample frequency (which is almost always 20 Hz) and the date_time of the first row in the block.

The date_time of the first block is the time of the GPS measurement preceding the accelerometer measurement. If there is no valid GPS measurement, this time is calculated from the last GPS measurement and the time of the internal clock of the tracker.

Therefore, to know all valid dates and times that mark a valid block for a tracker, you need to consider all of the values in column date_time in gps.ee_tracking_speed_limited for the given tracker, column date_time on gps.ee_acc_start_limited, and all the values on column date_time for that tracker.

This table structure has ''not'' changed with respect to our previous database.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker that reported the fix. Part of the unique identifier of the GPS fix.
|-
|date_time||timestamp||NO||Time of the acceleration block that this measurement belongs to (time in UTC, no time zone).
|-
|index||int2||NO||Sorting order of the measurement within the acceleration block (each block starts at 0).
|-
|x_acceleration||int2||YES||Acceleration measured on the x (or surge) axis (range between -2666 and 2666).
|-
|y_acceleration||int2||YES||Acceleration measured on the y (or sway) axis (range between -2666 and 2666).
|-
|z_acceleration||int2||YES||Acceleration measured on the z (or heave) axis (range between -2666 and 2666).
|}

=== ''gps.''ee_acc_start_limited ===

Each row represents the start of a block of acceleration measurements when a GPS fix is not available at that acceleration block's time.

If there is no valid GPS measurement, this time is calculated from the last GPS measurement and the time of the internal clock of the tracker.

To know all valid dates and times that mark a valid block for a tracker, you need to consider all of the values in column date_time on gps.ee_tracking_speed_limited for the given tracker, column date_time on gps.ee_acc_start_limited, and all the values on column date_time for that tracker. And the order is important within each block, so you should sort the acceleration measurements by block and index.

Added in firmware 1.0.2.7.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker that reported the acceleration measurement.
|-
|date_time||timestamp||NO||Time to mark the start of the acceleration block.
|-
|line_counter||int4||NO||Number of lines associated with this set of accelerations.
|-
|timesynced||int2||YES||The source of the time that marks the acceleration block (0=not synced, 1=user set, 2=Gps Synced).
|-
|ascii||int4||YES||Acceleration measurement interval.
|-
|accsn||int4||YES||Number of samples (0-65536).
|-
|f||int2||YES||Acceleration sampling frequency (0=20Hz, 1=10Hz, 2=5Hz, 3=1Hz).
|}

== Bird data ==
=== ''gps.''ee_individual_limited ===

Each row shows bird's details for all birds that your projects allow you to see.

This view replaces the old gps.uva_individual_limited.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|ring_number||varchar||NO||Ring number.
|-
|species_latin_name||varchar||NO||Latin name of the species of the bird.
|-
|colour_ring||varchar||YES||String for ring colour.
|-
|sex||varchar||NO||Sex (M=male, F=female, X=unknown/unspecified)
|-
|mass||numeric(5,0)||YES||Mass (grams)
|-
|start_date||timestamp||NO||Date when the bird was first tracked with an UvA-BiTS logger.
|-
|end_date||timestamp||NO||Date at which the bird is no longer relevant for the tracking project.
|-
|remarks||text||YES||Text for user remarks
|-
|individual_id||int8||NO||Bird unique id. Included just to enable joins.
|}

=== ''gps.''ee_species_limited ===

Each row represents a species that you can assign to your birds to include them in your project. No bird in the system can exist without being linked to one of the species from this list. If you are tagging a bird whose species does not exist in the list, please [http://www.uva-bits.nl/contact/ contact] us.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|abbr||varchar||NO||Unique abbreviation for this species.
|-
|english_name||varchar||YES||Name of the species in English.
|-
|latin_name||varchar||NO||Name of the species in Latin. It is always filled in.
|-
|species_id||int8||NO||Unique id. Added to allow joins.
|}

=== ''gps.''ee_nest_limited ===
This view shows nests you have access to. A nest is always for a bird with a tracker (so a track session must exist for that bird), and must contain a short name (a reference name for you to use) and a latitude and longitude pair.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|reference_name||varchar||NO||A string with a useful reference for you. We recommend its being unique for each nest in a project, but the system does NOT enforce it.
|-
|latitude||numeric||YES||Latitude of the coordinates where the nest was found, decimal degrees.
|-
|longitude||numeric||YES||Longitude of the coordinates where the nest was found, decimal degrees.
|-
|start_date_time||timestamp||NO||When the nest was spotted.
|-
|end_date_time||timestamp||NO||When the nest ceases to be relevant.
|-
|found_by_whom||varchar||YES||Place to credit people involved in finding the nest.
|-
|remarks||text||YES||Free text.
|-
|location||bytearray||YES||Calculated Geometry of the coordinates where the nest was found.
|-
|nest_id||int8||NO||Unique id. Added to allow joins.
|}

=== ''gps.''ee_nest_inhabitant_limited ===
This view shows who lives on which nest, for birds you have access to.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|reference_name||varchar||NO||A string belonging to the nest that you can use to identify that nest. Because it is NOT enforced to be unique, DO NOT USE THIS FIELD FOR JOINING.
|-
|ring_number||varchar||NO||The ring number of a bird on a nest.
|-
|device_info_serial||int8||NO||Following the link from a bird to her track sessions, and considering the nest's start and end time, the bird's trackers within that period are resolved so that they can be shown here. This is the device_info_serial of a tracker.
|-
|key_name||varchar||NO||Following the link from a bird to her track sessions, and considering the nest's start and end time, the track sessions' projects within that period are resolved so that they can be shown here. This is the key_name of a project.
|-
|nest_id||int8||NO||Reference to the nest id. Added to allow joins.
|-
|individual_id||int8||NO||Reference to the individual id. Added to allow joins.
|-
|track_session_id||int8||NO||Reference to a resolved track session id. Added to allow joins.
|}
== Tracker and project data ==

=== ''gps.''ee_project_limited ===

Each row shows the details of each project, for the projects you are allowed to see.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||Unique key name, with the form <species>_<place>
|-
|station_name||varchar||NO||Where the receiving station is
|-
|start_date||timestamp||NO||Project start date
|-
|end_date||timestamp||NO||Date at which the project is not relevant any more
|-
|description||text||YES||String for free description. Updatable by the user.
|-
|project_id||int8||NO||Project unique id. Included just to enable joins.
|-
|parent_id||int8||YES||Reference to the parent project in the hierarchy.
|}

=== ''gps.''ee_tracker_limited ===

Each row shows tracker details for all trackers that your projects allow you to see.

This view replaces the old gps.uva_device_limited.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int8||NO||Serial number of the tracker
|-
|firmware_version||varchar||YES||String for firmware version
|-
|mass||numeric(4,2)||YES||Mass (in grams)
|-
|start_date||timestamp||NO||Date before which the device is not relevant
|-
|end_date||timestamp||NO||Date at which the device is not relevant any more
|-
|x_o||numeric(30,6)||YES||Accelerometer calibration: offset on the x (or surge) axis
|-
|x_s||numeric(30,6)||YES||Accelerometer calibration: sensitivity on the x (or surge) axis
|-
|y_o||numeric(30,6)||YES||Accelerometer calibration: offset on the y (or sway) axis
|-
|y_s||numeric(30,6)||YES||Accelerometer calibration: sensitivity on the y (or sway) axis
|-
|z_o||numeric(30,6)||YES||Accelerometer calibration: offset on the z (or heave) axis
|-
|z_s||numeric(30,6)||YES||Accelerometer calibration: sensitivity on the y (or heave) axis
|-
|tracker_id||int8||NO||Tracker unique id. Included just to enable joins.
|}

=== ''gps.''ee_track_session_limited ===

A track session links a tracker and a bird for a given period of time. Therefore, it is '''mandatory for projects''' to supply this piece of '''metadata''' before they can see any tracking data related to a tracker they own.

In order to create a track session, the project must create first an entry in the system for the bird that they are attaching the tracker to, and this is the way for the UvA-BiTS community to make sense of the tracking data that we are receiving. This is why providing the metadata is so important.

By editing the start and end dates, administrators of the project that this track session belongs to, can delimit (and therefore, establish) when the coupling makes sense (or is to be understood as useful).

Whenever a tracker is attached to a bird, a track session must be started (start_date) to indicate that the tracking data that is received is already live data. If the tracker is separated from the bird, the tracking data ceases to make sense, so the track session's end date should be set to denote it has ended. New track sessions can be started if the tracker is re-attached to a bird (possibly the same as before, or maybe a different one) afterwards.

Each row shows track session's information. There exists a row for each track session that your projects allow you to see.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||The project that the track session belongs to.
|-
|device_info_serial||int8||NO||Serial number of the tracker this track session involves.
|-
|ring_number||varchar||NO||Ring number of the bird that this track session involves.
|-
|start_date||timestamp||NO||Date before which the track session is not relevant
|-
|end_date||timestamp||NO||Date at which the track session is not relevant any more
|-
|remarks||text||YES||Free text for comments
|-
|start_latitude||numeric(11,8)||NO||Latitude where the bird was tagged, decimal degrees
|-
|start_longitude||numeric(11,8)||NO||Longitude where the bird was tagged, decimal degrees
|-
|project_id||int8||NO||Reference to the project this track session belongs to. Included just to enable joins.
|-
|tracker_id||int8||NO||Reference to the tracker this track session involves. Included just to enable joins.
|-
|individual_id||int8||NO||Reference to the individual this track session involves. Included just to enable joins.
|-
|track_session_id||int8||NO||Unique identifier of the track session. Included just to enable joins.
|}

== Relation to projects ==

=== ''gps.''ee_user_limited ===

Each row shows a colleague in a project that you belong to, indicating her role and status there.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||Project name
|-
|email||varchar||NO||e-mail address
|-
|login_name||varchar||NO||Login name
|-
|name||varchar||NO||(first) name
|-
|surname||varchar||NO||Surname
|-
|role||varchar||NO||Role in the project
|-
|status||varchar||NO||Status of the membership
|-
|user_id||int8||NO||Unique identifier of the user. Included to enable joins.
|}

=== ''gps.''ee_tracker_ownership_limited ===

Each row that you see expresses that a tracker belongs to one of the projects you are involved with.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||The project that the tracker belongs to.
|-
|device_info_serial||int8||NO||Serial number of the tracker.
|-
|project_id||int8||NO||Reference to the project. Included just to enable joins.
|-
|tracker_id||int8||NO||Reference to the tracker. Included just to enable joins.
|}

=== ''gps.''ee_individual_ownership_limited ===

Each row that you see expresses that a bird belongs to one of the projects you are involved with.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||The project that the individual belongs to.
|-
|ring_number||varchar||NO||Ring number of the bird.
|-
|project_id||int8||NO||Reference to the project. Included just to enable joins.
|-
|individual_id||int8||NO||Reference to the individual. Included just to enable joins.
|}

== Engineering data ==

'''Note:''' These views will show no rows for trackers not yet attached to birds.

=== ''gps.''ee_comm_limited ===

Each row indicates status and result of communications with ground stations for each tracker that your project allows you to see.

Only for newer firmware (v.1.0.1.4 and higher).

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
| device_info_serial || int4 || NO || Serial number of the tracker.
|-
| date_time || timestamp || NO || timestamp of start communication
|-
| date_time_end || timestamp ||YES|| timestamp of end communication
|-
| date_time_utc || timestamp ||YES|| date_time in utc (if available)
|-
| date_time_end_utc || timestamp ||YES|| date_time_end in utc (if available)
|-
| action || text||YES|| JOINED / RESCHEDULE ??
|-
| bytes_communicated || int4 ||YES|| Number of bytes communicated (total of all bytes per DATA line)
|-
|status || text ||YES|| READY, TIMEOUT, LOWPOWER, MISSING
|-
| network_join_time || int4||YES||milliseconds
|-
| number_identical_lines || int4 ||YES|| Number identical lines in data block. Gives indication
|-
| non_ascii_char || int4 ||YES|| Number of non ascii characters (indicated by question-mark ?)
|-
| sts_time || timestamp ||YES|| derived from S0 hh,mi
|-
| timesynced || int2||YES|| TimeSynced (0=not sy:wnced, 1=user set, 2=Gps Synced)
|-
|devicemode || int2||YES||DeviceMode (0-99). This describes whether the device is in a special operational mode. 0 = normal operational mode; 80 hex = Memory save mode (memory is almost full and all activity is drastically reduced so the circular memory probably won't be overwritten or, if it does, it will be kept to a minimum)
|-
| vbat || float8 ||YES|| Battery voltage in V (0-~4.200V)
|-
|logb || int4 ||YES|| logged bytes, waiting for download (0-4Mbyte)
|-
| logp || int2 ||YES|| Logged percentage (units 0.1%) of memory waiting to be downloaded (0-1000)
|-
| gpsii || int4 ||YES|| GpsIval, current (0-65535)
|-
|comii || int4||YES|| ComIval, current (0-65535)
|-
|accii || int4||YES|| AcceleroIval, current (0-65535)
|-
|swddd || int4||YES|| User switches, current (0-65535)
|-
|last_gps_fix ||timestamp||YES||timestamp of last fix
|-
|latitude||float8||YES||Latitude of last fix. Decimal degrees
|-
|longitude||float8||YES||Longitude of last fix. Decimal degrees
|}

=== ''gps.''ee_energy_limited ===

Only for newer firmware.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker
|-
|date_time||timestamp||NO||Date and time of the reported data
|-
|timesynced ||int2||YES||TimeSynced (0=not synced, 1=user set, 2=Gps synced)
|-
|devicemode ||int2||YES||(0-99)
|-
|vsll ||float8||YES||Solar Voltage during charging in mV (0-~2500)
|-
|vbat ||float8||YES||Battery Voltage in V (0-~4.200)
|-
|ssw ||int2||YES|| Solar Cell Switches ... (0-255) fw1022 and up
|-
| ||||YES||extended fields if special user switch is on
|-
|vsoo ||float8||YES|| Solar voltage open in V (0-~2.500)
|-
|vdrf ||float8||YES||reference voltage Dac in V (0-~2.435)
|-
|vchr ||float8||YES||charge current voltage
|-
|g ||int2||YES|| Gps battery threshold OK
|-
|c || int2||YES||Com battery threshold OK
|-
|m || int2||YES||Charge Mode (0,1,2 Com battery threshold OK
|-
|pressure || int4||YES|| in pascal (1mBar = 100Pa, 1pressuremeter = ~12Pa)
|-
|temperature || float8||YES|| in celcius degrees
|}

=== ''gps.''ee_settings_limited ===

Settings are either from C0-C1-C9 lines or parsed from uplink communication lines.
If C0-C1 is parsed its values are compared with the previous value and if the settings are the same, then no new records need to be added.
If uplink settings are parsed, then wait until the last setting has been seen (DL command or EOF); then store all the new settings in one record. Only for newer firmware.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||
|-
|date_time||timestamp||NO||Derived timestamp from Hh:mi fields in C0 combined with computer timestamp in JOINED line
|-
|timesynced || int2 ||YES||
|-
|timezone_time || int2 ||YES||
|-
|swddd || int4||YES|| SpecialUserSwitches
|-
|gpsnavmode || int2||YES|| GpsNavMode 0-8. This is the Kalman filter used for GPS navigation mode often called the "Dynamic platform model". The following codes are provided by the GPS chip manufacturer for the UBlox GPS chip LEAS4S: 1 = stationary, 2 = pedestrian, 3 = automotive, 4 = sea, 5 = airborne 1G, 6 = airborne 2G, 7 = airborne 3G. The UBlox GPS chip LEAS6S uses the following codes: 0 = portable, 2 = stationary, 3 = pedestrian, 4 = automotive, 5 = sea, 6 = airborne with < 1g acceleration, 7 = airborne with < 2g acceleration, 8 = airborne with < 4g acceleration
|-
|gpsreceivermode || int2 ||YES|| GpsReceiverNode 0-3
|-
|gpsfixtime || int2||YES|| GpsFixTime 0-255
|-
|gps_ivaln_on_h || int2||YES|| hours Gps IvalN on
|-
|gps_ivaln_on_m || int2||YES|| minutes Gps IvalN on
|-
|gps_ivaln || int4||YES|| GpsIvalN (0-65530)
|-
|gps_ivale_on_h || int2||YES|| hours Gps IvalE on
|-
|gps_ivale_on_m || int2||YES|| minutes GpS IvalE on
|-
|gps_ivale || int4||YES|| GpsIvalE
|-
|com_ivaln_on_h || int2||YES|| hours Com IvalN on
|-
|com_ivaln_on_m || int2||YES|| minutes Com IvalN on
|-
|com_ivaln || int4||YES|| ComIvalN (0-65530)
|-
|com_ivale_on_h || int2||YES|| hours Com IvalE on
|-
|com_ivale_on_m || int2||YES|| minutes Com IvalE on
|-
|com_ivale || int4||YES|| ComIvalE
|-
|accii || int4||YES|| Accelero measurement interval
|-
|accbn || int4||YES|| Accelero number of blocks
|-
|accsn || int4||YES|| Accelero number of samples (0-65536)
|-
|f || int2||YES||Accelero sample frequency (0=20Hz, 1=10Hz, 2=5Hz, 3=1Hz)
|-
|pdp || int2||YES|| PdopMsk (3-250)
|-
|pac||int2||YES|| PaccMsk (10-100)
|-
|tdp ||int2||YES|| TdopMsk (3-250)
|-
|tac ||int2||YES|| TaccMsk (30-300)
|-
|ga || int2||YES|| Additional Fixes GpsIval mode (3-25) before interval fix is logged
|-
|aol || int2||YES||AccMemOffLev (100 - 950) units 0.1%; if Memfull above this level all Accelero activity will be overruled and switched off untill MemFull < level - 5
|-
|eventnum || text||YES|| 12 eventnumbers (0-0xff) only printed if not zero, therfore converted to text
|}

=== ''gps.''ee_trackingfile_limited ===

Each row shows the status of the last attempt to parse (process) the raw data of a file for a given tracker, for each tracker that your projects allow you to see.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|id||int8||NO||Unique identifier of the row
|-
|date_reported||date||NO||Date that the file arrived
|-
|filename||varchar||NO||Name of the file
|-
|last_modified_date||timestamp||NO||Date that the file was last modified on
|-
|device_info_serial||int4||YES||Serial number of the tracker this file comes from
|-
|filesize||int8||NO||Size of the file in KB
|-
|parsed_date||timestamp||YES||Date and time that the file was last attempted to be parsed on
|-
|version||int4||NO||Optimistic locking field. Ignore it as a user.
|-
|last_found_date||timestamp||YES||Last time that a process that runs to look for new files saw the file
|-
|n_lines||int4||YES||Count of lines found in the file last time it was processed
|-
|n_data_lines||int4||YES||Count of actual data lines, out of the total lines
|-
|n_tracking_records||int4||YES||Count of GPS fixes found during the parsing process
|-
|n_communication_records||int4||YES||Count of communication records found during the parsing process
|-
|n_energy_records||int4||YES||Count of energy records found during the parsing process
|-
|n_acceleration_records||int4||YES||Count of acceleration records found during the parsing process
|-
|n_acceleration_start_records||int4||YES||Count of acceleration start records found during the parsing process
|-
|n_errors||int4||YES||Count of errors encountered during the parsing process
|-
|n_settings_records||int4||YES||Count of settings records found during the parsing process
|-
|filepath||text||NO||Directory path of the file on the server
|}

=== ''gps.''ee_warning_limited ===

Since firmware version 1.0.2.7 this table was added for events that do not occur very often (C10, C20) or errors that occur during parsing.

For example all CRC failures can be selected with :

select * from gps.ee_warning_limited
where warning = 'CRC8 failure'
order by device_info_serial;

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker
|-
|filename||text||NO||File name
|-
|linenumber||integer||YES||line number in file
|-
|warning||text||YES||The warning message
|-
|line||text||YES||The actual line contents in the file
|}

== Sharings ==

Projects can now share trackers with other projects during periods of time. This means that people in the receiving projects can access information that these shared trackers provide during those time periods.

The following views allow you to see this shared information:

* Tracking data
** ''gps.''ee_shared_tracking_speed_limited: tracking information (gps, speed), during the shared period, for trackers shared
** ''gps.''ee_shared_acc_start_limited: acceleration start information, during the shared period, for trackers shared
** ''gps.''ee_shared_acceleration_limited: acceleration information for trackers shared
* ''Meta'' data
** ''gps.''ee_shared_tracker_limited: information about each shared tracker
** ''gps.''ee_shared_individual_limited: information about birds with shared trackers
** ''gps.''ee_shared_track_session_limited: information about track sessions which involve shared trackers
* Relation to projects
** ''gps.''ee_shared_tracker_sharing_limited: view information for each sharing that your projects allow you to see

== Accelerometer calibration ==

The accelerometer measures the acceleration in 3D (20 times per second). Acceleration is caused by gravity (static component) and by a change in speed (dynamic component).
When the logger is static in horizontal position (solar cells up), gravity (green arrow) shows a heave (vertical acceleration, z) of +1 g (1 g = 9.81 m.s-2) while surge and heave are both 0. When the front of the logger is vertically pointing down (forward acceleration, x), surge is +1g and when it is rolled to the right (sidewards acceleration, y) sway is +1g. Due to gravity, the pitch of a logger is thus translated into the values of heave, surge and sway (or z, x, y).

However, the values are measured in mV instead of g. Before delivering the GPS-loggers, they have been calibrated at UvA. The acceleration (A) expressed in g is calculated from the measured value (M) and the calibration factors offset (''O'') and sensitivity (''S'').

Thus acceleration for heave (z), surge (x) and sway (y) is calculated as:
Az = (Mz-Oz)/Sz ; Ax = (Mx-Ox)/Sx; Ay = (My-Oy)/Sy ;

The values of ''O'' (in general ranging from 0-60, sometimes even 300) and ''S'' (ranging from 1300-1450) are available in the device table of the data base. If not available, use the factory defaults ''O'' = 0 and ''S'' = 1365.

{|
| [[File:accelero1.png|300px]]||[[File:accelero2.png|300px]]|
|}

In the figure above heave (z) is shown in green, surge(x) in red and sway (y) in blue, all values are expressed in g (on the y-axis).

== Data processing and filtering ==
The raw log files are processed before being entered into the database. Each firmware version has a dedicated parser for the task. In certain cases data that is found in the log files is not stored in the database. This can occur for two reasons:

1. The data did not pass the CRC8 checksum (cyclic redundancy check). This is a standard procedure designed to detect accidental changes to raw computer data.

2. If the data has an invalid signature. For example, if a certain data format (eg numeric) or number of values (eg 4 digits) is expected but not returned, this record is not stored in the database.

When a line is deleted, the log file name, line number and the type of error is stored in the table gps.uva_warning102

== Information about GPS receivers ==
The first series of tags used the UBlox LEAS4S chip (~up to tag 541), newer tags use UBlox LEAS6S. The series developed in 2012 will use the Max6G receivers. From 2012 receiver versions will be stored in the database.
The UBX protocol is used for programming firmware, packaging and processing data.
For more information about the receivers themselves see the ublox documentation.
Ublox 4 Protocol specification [http://www.u-blox.com/images/downloads/Product_Docs/ANTARIS_Protocol_Specification%28GPS.G3-X-03002%29.zip]
Ublox 6 Protocol specification [http://www.u-blox.com/images/downloads/Product_Docs/u-blox6_ReceiverDescriptionProtocolSpec_%28GPS.G6-SW-10018%29.pdf]

UvA-BiTS Tracking Data

2014-12-11T13:43:47Z

Judy: /* gps.ee_acceleration_limited */

== Tracking data ==

'''Note:''' These views will show no rows for trackers not yet attached to birds.

=== ''gps.''ee_tracking_speed_limited ===

Shows the tracking data that your projects allow you to see.

Each row represents a GPS fix as it is interpreted from the data that comes via the GPS tracker, plus it adds speeds.

It replaces old views gps.uva_tracking_limited, gps.uva_tracking_speed_3d_limited and gps.uva_tracking_speed. This table structure has ''not'' changed with respect to our previous database.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker that reported the fix. Part of the unique identifier of the GPS fix.
|-
|date_time||timestamp||NO||Time when the fix was taken. UTC (i.e.: no timezone). Part of the unique identifier of the GPS fix.
|-
|latitude||float8||YES||Latitude of the GPS fix, decimal degrees.
|-
|longitude||float8||YES||Longitude of the GPS fix, decimal degrees.
|-
|altitude||int4||YES||Altitude above sea level measured by GPS tag in meters
|-
|pressure||int4||YES||Pressure measured by GPS tag sensor in Pascals
|-
|temperature||float8||YES||Temperature measured by GPS tag sensor in Celsius
|-
|h_accuracy||float8||YES||Horizontal accuracy (meters)
|-
|v_accuracy||float8||YES||vertical accuracy (meters)
|-
|x_speed||float8||YES||meters/second
|-
|y_speed||float8||YES||meters/second
|-
|z_speed||float8||YES||meters/second
|-
|gps_fixtime||float8||YES||GPS fix time (also known as Time to Fix) in seconds
|-
|userflag||int4||YES||Acceptable = 0; Data flagged as unacceptable by user if not equal to 0.
|-
|satellites_used||int2||YES||Number of satellites used for fix the GPS measurement
|-
|positiondop||float8||YES||Positional DOP (Dilution of Precision), unitless
|-
|speed_accuracy||float8||YES||
|-
|location||geometry||YES||postgreSQL geometry; to allow GIS operations
|-
|vnorth||float8||YES||- vx * sin(lat) * cos(long) - vy * sin(lat) * sin (long) + vz * cos(lat), m/s
|-
|veast||float8||YES||- vx * sin(lon) + vy * cos(lon), m/s
|-
|vdown||float8||YES||- vx * cos(lat) * cos(lon) - vy * cos(lat) * sin(lon) - vz * sin(lat), m/s
|-
|speed||float8||YES||- vx * sin(lon) + vy * cos(lon)
|-
|speed_3d||float8||YES|| sqrt(d.x_speed^2 + d.y_speed^2 + d.z_speed^2), m/s
|-
|speed_2d||float8||YES|| Speed on 2D, m/s
|-
|direction||numeric||YES|| Heading in degrees; 0 is north, 90 is east
|-
|altitude_agl||float8||YES|| Reported GPS altitude MINUS terrain elevation on the coordinate based on Digital Elevation Model data used in the e-ecology database (for further information see: [[Elevation]]
|}

=== ''gps.''ee_acceleration_limited ===

Acceleration is measured in blocks of 1-200 measurements (records). The date_time mentioned in this block is the date and time of the first acceleration measurement. Sequential measurements within a block are numbered with an index 1... (max 200). Every row represents one measurement in 3 directions (X,Y,Z). The order is important within each block, so you should always sort the acceleration measurements by block and index. If you need the exact time for each row, you can calculate it from the index, the sample frequency (which is almost always 20 Hz) and the date_time of the first row in the block.

The date_time of the first block is the time of the GPS measurement preceding the accelerometer measurement. If there is no valid GPS measurement, this time is calculated from the last GPS measurement and the time of the internal clock of the tracker.

Therefore, to know all valid dates and times that mark a valid block for a tracker, you need to consider all of the values in column date_time in gps.ee_tracking_speed_limited for the given tracker, column date_time on gps.ee_acc_start_limited, and all the values on column date_time for that tracker.

This table structure has ''not'' changed with respect to our previous database.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker that reported the fix. Part of the unique identifier of the GPS fix.
|-
|date_time||timestamp||NO||Time of the acceleration block that this measurement belongs to (time in UTC, no time zone).
|-
|index||int2||NO||Sorting order of the measurement within the acceleration block (each block starts at 0).
|-
|x_acceleration||int2||YES||Acceleration measured on the x (or surge) axis (range between -2666 and 2666).
|-
|y_acceleration||int2||YES||Acceleration measured on the y (or sway) axis (range between -2666 and 2666).
|-
|z_acceleration||int2||YES||Acceleration measured on the z (or heave) axis (range between -2666 and 2666).
|}

=== ''gps.''ee_acc_start_limited ===

Each row represents the start of a block of acceleration measurements when a GPS fix is not available at that acceleration block's time.

The logger tries to mark the start of blocks from the GPS fix that coincides with the begin of the acceleration block. If there's no GPS fix during that block, then it extrapolates the date and time and makes a special record which is interpreted by our accelerometer processing software and made available through this view.

Therefore, to know all valid dates and times that mark a valid block for a tracker, you need to consider all of the values in column date_time on gps.ee_tracking_speed_limited for the given tracker, column date_time on gps.ee_acc_start_limited, and all the values on column date_time for that tracker. And the order is important within each block, so you should sort the acceleration measurements by block and index.

Added in firmware 1.0.2.7.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker that reported the acceleration measurement.
|-
|date_time||timestamp||NO||Time to mark the start of the acceleration block.
|-
|line_counter||int4||NO||Number of lines associated with this set of accelerations.
|-
|timesynced||int2||YES||The source of the time that marks the acceleration block (0=not synced, 1=user set, 2=Gps Synced).
|-
|ascii||int4||YES||Acceleration measurement interval.
|-
|accsn||int4||YES||Number of samples (0-65536).
|-
|f||int2||YES||Acceleration sampling frequency (0=20Hz, 1=10Hz, 2=5Hz, 3=1Hz).
|}

== Bird data ==
=== ''gps.''ee_individual_limited ===

Each row shows bird's details for all birds that your projects allow you to see.

This view replaces the old gps.uva_individual_limited.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|ring_number||varchar||NO||Ring number.
|-
|species_latin_name||varchar||NO||Latin name of the species of the bird.
|-
|colour_ring||varchar||YES||String for ring colour.
|-
|sex||varchar||NO||Sex (M=male, F=female, X=unknown/unspecified)
|-
|mass||numeric(5,0)||YES||Mass (grams)
|-
|start_date||timestamp||NO||Date when the bird was first tracked with an UvA-BiTS logger.
|-
|end_date||timestamp||NO||Date at which the bird is no longer relevant for the tracking project.
|-
|remarks||text||YES||Text for user remarks
|-
|individual_id||int8||NO||Bird unique id. Included just to enable joins.
|}

=== ''gps.''ee_species_limited ===

Each row represents a species that you can assign to your birds to include them in your project. No bird in the system can exist without being linked to one of the species from this list. If you are tagging a bird whose species does not exist in the list, please [http://www.uva-bits.nl/contact/ contact] us.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|abbr||varchar||NO||Unique abbreviation for this species.
|-
|english_name||varchar||YES||Name of the species in English.
|-
|latin_name||varchar||NO||Name of the species in Latin. It is always filled in.
|-
|species_id||int8||NO||Unique id. Added to allow joins.
|}

=== ''gps.''ee_nest_limited ===
This view shows nests you have access to. A nest is always for a bird with a tracker (so a track session must exist for that bird), and must contain a short name (a reference name for you to use) and a latitude and longitude pair.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|reference_name||varchar||NO||A string with a useful reference for you. We recommend its being unique for each nest in a project, but the system does NOT enforce it.
|-
|latitude||numeric||YES||Latitude of the coordinates where the nest was found, decimal degrees.
|-
|longitude||numeric||YES||Longitude of the coordinates where the nest was found, decimal degrees.
|-
|start_date_time||timestamp||NO||When the nest was spotted.
|-
|end_date_time||timestamp||NO||When the nest ceases to be relevant.
|-
|found_by_whom||varchar||YES||Place to credit people involved in finding the nest.
|-
|remarks||text||YES||Free text.
|-
|location||bytearray||YES||Calculated Geometry of the coordinates where the nest was found.
|-
|nest_id||int8||NO||Unique id. Added to allow joins.
|}

=== ''gps.''ee_nest_inhabitant_limited ===
This view shows who lives on which nest, for birds you have access to.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|reference_name||varchar||NO||A string belonging to the nest that you can use to identify that nest. Because it is NOT enforced to be unique, DO NOT USE THIS FIELD FOR JOINING.
|-
|ring_number||varchar||NO||The ring number of a bird on a nest.
|-
|device_info_serial||int8||NO||Following the link from a bird to her track sessions, and considering the nest's start and end time, the bird's trackers within that period are resolved so that they can be shown here. This is the device_info_serial of a tracker.
|-
|key_name||varchar||NO||Following the link from a bird to her track sessions, and considering the nest's start and end time, the track sessions' projects within that period are resolved so that they can be shown here. This is the key_name of a project.
|-
|nest_id||int8||NO||Reference to the nest id. Added to allow joins.
|-
|individual_id||int8||NO||Reference to the individual id. Added to allow joins.
|-
|track_session_id||int8||NO||Reference to a resolved track session id. Added to allow joins.
|}
== Tracker and project data ==

=== ''gps.''ee_project_limited ===

Each row shows the details of each project, for the projects you are allowed to see.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||Unique key name, with the form <species>_<place>
|-
|station_name||varchar||NO||Where the receiving station is
|-
|start_date||timestamp||NO||Project start date
|-
|end_date||timestamp||NO||Date at which the project is not relevant any more
|-
|description||text||YES||String for free description. Updatable by the user.
|-
|project_id||int8||NO||Project unique id. Included just to enable joins.
|-
|parent_id||int8||YES||Reference to the parent project in the hierarchy.
|}

=== ''gps.''ee_tracker_limited ===

Each row shows tracker details for all trackers that your projects allow you to see.

This view replaces the old gps.uva_device_limited.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int8||NO||Serial number of the tracker
|-
|firmware_version||varchar||YES||String for firmware version
|-
|mass||numeric(4,2)||YES||Mass (in grams)
|-
|start_date||timestamp||NO||Date before which the device is not relevant
|-
|end_date||timestamp||NO||Date at which the device is not relevant any more
|-
|x_o||numeric(30,6)||YES||Accelerometer calibration: offset on the x (or surge) axis
|-
|x_s||numeric(30,6)||YES||Accelerometer calibration: sensitivity on the x (or surge) axis
|-
|y_o||numeric(30,6)||YES||Accelerometer calibration: offset on the y (or sway) axis
|-
|y_s||numeric(30,6)||YES||Accelerometer calibration: sensitivity on the y (or sway) axis
|-
|z_o||numeric(30,6)||YES||Accelerometer calibration: offset on the z (or heave) axis
|-
|z_s||numeric(30,6)||YES||Accelerometer calibration: sensitivity on the y (or heave) axis
|-
|tracker_id||int8||NO||Tracker unique id. Included just to enable joins.
|}

=== ''gps.''ee_track_session_limited ===

A track session links a tracker and a bird for a given period of time. Therefore, it is '''mandatory for projects''' to supply this piece of '''metadata''' before they can see any tracking data related to a tracker they own.

In order to create a track session, the project must create first an entry in the system for the bird that they are attaching the tracker to, and this is the way for the UvA-BiTS community to make sense of the tracking data that we are receiving. This is why providing the metadata is so important.

By editing the start and end dates, administrators of the project that this track session belongs to, can delimit (and therefore, establish) when the coupling makes sense (or is to be understood as useful).

Whenever a tracker is attached to a bird, a track session must be started (start_date) to indicate that the tracking data that is received is already live data. If the tracker is separated from the bird, the tracking data ceases to make sense, so the track session's end date should be set to denote it has ended. New track sessions can be started if the tracker is re-attached to a bird (possibly the same as before, or maybe a different one) afterwards.

Each row shows track session's information. There exists a row for each track session that your projects allow you to see.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||The project that the track session belongs to.
|-
|device_info_serial||int8||NO||Serial number of the tracker this track session involves.
|-
|ring_number||varchar||NO||Ring number of the bird that this track session involves.
|-
|start_date||timestamp||NO||Date before which the track session is not relevant
|-
|end_date||timestamp||NO||Date at which the track session is not relevant any more
|-
|remarks||text||YES||Free text for comments
|-
|start_latitude||numeric(11,8)||NO||Latitude where the bird was tagged, decimal degrees
|-
|start_longitude||numeric(11,8)||NO||Longitude where the bird was tagged, decimal degrees
|-
|project_id||int8||NO||Reference to the project this track session belongs to. Included just to enable joins.
|-
|tracker_id||int8||NO||Reference to the tracker this track session involves. Included just to enable joins.
|-
|individual_id||int8||NO||Reference to the individual this track session involves. Included just to enable joins.
|-
|track_session_id||int8||NO||Unique identifier of the track session. Included just to enable joins.
|}

== Relation to projects ==

=== ''gps.''ee_user_limited ===

Each row shows a colleague in a project that you belong to, indicating her role and status there.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||Project name
|-
|email||varchar||NO||e-mail address
|-
|login_name||varchar||NO||Login name
|-
|name||varchar||NO||(first) name
|-
|surname||varchar||NO||Surname
|-
|role||varchar||NO||Role in the project
|-
|status||varchar||NO||Status of the membership
|-
|user_id||int8||NO||Unique identifier of the user. Included to enable joins.
|}

=== ''gps.''ee_tracker_ownership_limited ===

Each row that you see expresses that a tracker belongs to one of the projects you are involved with.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||The project that the tracker belongs to.
|-
|device_info_serial||int8||NO||Serial number of the tracker.
|-
|project_id||int8||NO||Reference to the project. Included just to enable joins.
|-
|tracker_id||int8||NO||Reference to the tracker. Included just to enable joins.
|}

=== ''gps.''ee_individual_ownership_limited ===

Each row that you see expresses that a bird belongs to one of the projects you are involved with.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||The project that the individual belongs to.
|-
|ring_number||varchar||NO||Ring number of the bird.
|-
|project_id||int8||NO||Reference to the project. Included just to enable joins.
|-
|individual_id||int8||NO||Reference to the individual. Included just to enable joins.
|}

== Engineering data ==

'''Note:''' These views will show no rows for trackers not yet attached to birds.

=== ''gps.''ee_comm_limited ===

Each row indicates status and result of communications with ground stations for each tracker that your project allows you to see.

Only for newer firmware (v.1.0.1.4 and higher).

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
| device_info_serial || int4 || NO || Serial number of the tracker.
|-
| date_time || timestamp || NO || timestamp of start communication
|-
| date_time_end || timestamp ||YES|| timestamp of end communication
|-
| date_time_utc || timestamp ||YES|| date_time in utc (if available)
|-
| date_time_end_utc || timestamp ||YES|| date_time_end in utc (if available)
|-
| action || text||YES|| JOINED / RESCHEDULE ??
|-
| bytes_communicated || int4 ||YES|| Number of bytes communicated (total of all bytes per DATA line)
|-
|status || text ||YES|| READY, TIMEOUT, LOWPOWER, MISSING
|-
| network_join_time || int4||YES||milliseconds
|-
| number_identical_lines || int4 ||YES|| Number identical lines in data block. Gives indication
|-
| non_ascii_char || int4 ||YES|| Number of non ascii characters (indicated by question-mark ?)
|-
| sts_time || timestamp ||YES|| derived from S0 hh,mi
|-
| timesynced || int2||YES|| TimeSynced (0=not sy:wnced, 1=user set, 2=Gps Synced)
|-
|devicemode || int2||YES||DeviceMode (0-99). This describes whether the device is in a special operational mode. 0 = normal operational mode; 80 hex = Memory save mode (memory is almost full and all activity is drastically reduced so the circular memory probably won't be overwritten or, if it does, it will be kept to a minimum)
|-
| vbat || float8 ||YES|| Battery voltage in V (0-~4.200V)
|-
|logb || int4 ||YES|| logged bytes, waiting for download (0-4Mbyte)
|-
| logp || int2 ||YES|| Logged percentage (units 0.1%) of memory waiting to be downloaded (0-1000)
|-
| gpsii || int4 ||YES|| GpsIval, current (0-65535)
|-
|comii || int4||YES|| ComIval, current (0-65535)
|-
|accii || int4||YES|| AcceleroIval, current (0-65535)
|-
|swddd || int4||YES|| User switches, current (0-65535)
|-
|last_gps_fix ||timestamp||YES||timestamp of last fix
|-
|latitude||float8||YES||Latitude of last fix. Decimal degrees
|-
|longitude||float8||YES||Longitude of last fix. Decimal degrees
|}

=== ''gps.''ee_energy_limited ===

Only for newer firmware.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker
|-
|date_time||timestamp||NO||Date and time of the reported data
|-
|timesynced ||int2||YES||TimeSynced (0=not synced, 1=user set, 2=Gps synced)
|-
|devicemode ||int2||YES||(0-99)
|-
|vsll ||float8||YES||Solar Voltage during charging in mV (0-~2500)
|-
|vbat ||float8||YES||Battery Voltage in V (0-~4.200)
|-
|ssw ||int2||YES|| Solar Cell Switches ... (0-255) fw1022 and up
|-
| ||||YES||extended fields if special user switch is on
|-
|vsoo ||float8||YES|| Solar voltage open in V (0-~2.500)
|-
|vdrf ||float8||YES||reference voltage Dac in V (0-~2.435)
|-
|vchr ||float8||YES||charge current voltage
|-
|g ||int2||YES|| Gps battery threshold OK
|-
|c || int2||YES||Com battery threshold OK
|-
|m || int2||YES||Charge Mode (0,1,2 Com battery threshold OK
|-
|pressure || int4||YES|| in pascal (1mBar = 100Pa, 1pressuremeter = ~12Pa)
|-
|temperature || float8||YES|| in celcius degrees
|}

=== ''gps.''ee_settings_limited ===

Settings are either from C0-C1-C9 lines or parsed from uplink communication lines.
If C0-C1 is parsed its values are compared with the previous value and if the settings are the same, then no new records need to be added.
If uplink settings are parsed, then wait until the last setting has been seen (DL command or EOF); then store all the new settings in one record. Only for newer firmware.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||
|-
|date_time||timestamp||NO||Derived timestamp from Hh:mi fields in C0 combined with computer timestamp in JOINED line
|-
|timesynced || int2 ||YES||
|-
|timezone_time || int2 ||YES||
|-
|swddd || int4||YES|| SpecialUserSwitches
|-
|gpsnavmode || int2||YES|| GpsNavMode 0-8. This is the Kalman filter used for GPS navigation mode often called the "Dynamic platform model". The following codes are provided by the GPS chip manufacturer for the UBlox GPS chip LEAS4S: 1 = stationary, 2 = pedestrian, 3 = automotive, 4 = sea, 5 = airborne 1G, 6 = airborne 2G, 7 = airborne 3G. The UBlox GPS chip LEAS6S uses the following codes: 0 = portable, 2 = stationary, 3 = pedestrian, 4 = automotive, 5 = sea, 6 = airborne with < 1g acceleration, 7 = airborne with < 2g acceleration, 8 = airborne with < 4g acceleration
|-
|gpsreceivermode || int2 ||YES|| GpsReceiverNode 0-3
|-
|gpsfixtime || int2||YES|| GpsFixTime 0-255
|-
|gps_ivaln_on_h || int2||YES|| hours Gps IvalN on
|-
|gps_ivaln_on_m || int2||YES|| minutes Gps IvalN on
|-
|gps_ivaln || int4||YES|| GpsIvalN (0-65530)
|-
|gps_ivale_on_h || int2||YES|| hours Gps IvalE on
|-
|gps_ivale_on_m || int2||YES|| minutes GpS IvalE on
|-
|gps_ivale || int4||YES|| GpsIvalE
|-
|com_ivaln_on_h || int2||YES|| hours Com IvalN on
|-
|com_ivaln_on_m || int2||YES|| minutes Com IvalN on
|-
|com_ivaln || int4||YES|| ComIvalN (0-65530)
|-
|com_ivale_on_h || int2||YES|| hours Com IvalE on
|-
|com_ivale_on_m || int2||YES|| minutes Com IvalE on
|-
|com_ivale || int4||YES|| ComIvalE
|-
|accii || int4||YES|| Accelero measurement interval
|-
|accbn || int4||YES|| Accelero number of blocks
|-
|accsn || int4||YES|| Accelero number of samples (0-65536)
|-
|f || int2||YES||Accelero sample frequency (0=20Hz, 1=10Hz, 2=5Hz, 3=1Hz)
|-
|pdp || int2||YES|| PdopMsk (3-250)
|-
|pac||int2||YES|| PaccMsk (10-100)
|-
|tdp ||int2||YES|| TdopMsk (3-250)
|-
|tac ||int2||YES|| TaccMsk (30-300)
|-
|ga || int2||YES|| Additional Fixes GpsIval mode (3-25) before interval fix is logged
|-
|aol || int2||YES||AccMemOffLev (100 - 950) units 0.1%; if Memfull above this level all Accelero activity will be overruled and switched off untill MemFull < level - 5
|-
|eventnum || text||YES|| 12 eventnumbers (0-0xff) only printed if not zero, therfore converted to text
|}

=== ''gps.''ee_trackingfile_limited ===

Each row shows the status of the last attempt to parse (process) the raw data of a file for a given tracker, for each tracker that your projects allow you to see.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|id||int8||NO||Unique identifier of the row
|-
|date_reported||date||NO||Date that the file arrived
|-
|filename||varchar||NO||Name of the file
|-
|last_modified_date||timestamp||NO||Date that the file was last modified on
|-
|device_info_serial||int4||YES||Serial number of the tracker this file comes from
|-
|filesize||int8||NO||Size of the file in KB
|-
|parsed_date||timestamp||YES||Date and time that the file was last attempted to be parsed on
|-
|version||int4||NO||Optimistic locking field. Ignore it as a user.
|-
|last_found_date||timestamp||YES||Last time that a process that runs to look for new files saw the file
|-
|n_lines||int4||YES||Count of lines found in the file last time it was processed
|-
|n_data_lines||int4||YES||Count of actual data lines, out of the total lines
|-
|n_tracking_records||int4||YES||Count of GPS fixes found during the parsing process
|-
|n_communication_records||int4||YES||Count of communication records found during the parsing process
|-
|n_energy_records||int4||YES||Count of energy records found during the parsing process
|-
|n_acceleration_records||int4||YES||Count of acceleration records found during the parsing process
|-
|n_acceleration_start_records||int4||YES||Count of acceleration start records found during the parsing process
|-
|n_errors||int4||YES||Count of errors encountered during the parsing process
|-
|n_settings_records||int4||YES||Count of settings records found during the parsing process
|-
|filepath||text||NO||Directory path of the file on the server
|}

=== ''gps.''ee_warning_limited ===

Since firmware version 1.0.2.7 this table was added for events that do not occur very often (C10, C20) or errors that occur during parsing.

For example all CRC failures can be selected with :

select * from gps.ee_warning_limited
where warning = 'CRC8 failure'
order by device_info_serial;

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker
|-
|filename||text||NO||File name
|-
|linenumber||integer||YES||line number in file
|-
|warning||text||YES||The warning message
|-
|line||text||YES||The actual line contents in the file
|}

== Sharings ==

Projects can now share trackers with other projects during periods of time. This means that people in the receiving projects can access information that these shared trackers provide during those time periods.

The following views allow you to see this shared information:

* Tracking data
** ''gps.''ee_shared_tracking_speed_limited: tracking information (gps, speed), during the shared period, for trackers shared
** ''gps.''ee_shared_acc_start_limited: acceleration start information, during the shared period, for trackers shared
** ''gps.''ee_shared_acceleration_limited: acceleration information for trackers shared
* ''Meta'' data
** ''gps.''ee_shared_tracker_limited: information about each shared tracker
** ''gps.''ee_shared_individual_limited: information about birds with shared trackers
** ''gps.''ee_shared_track_session_limited: information about track sessions which involve shared trackers
* Relation to projects
** ''gps.''ee_shared_tracker_sharing_limited: view information for each sharing that your projects allow you to see

== Accelerometer calibration ==

The accelerometer measures the acceleration in 3D (20 times per second). Acceleration is caused by gravity (static component) and by a change in speed (dynamic component).
When the logger is static in horizontal position (solar cells up), gravity (green arrow) shows a heave (vertical acceleration, z) of +1 g (1 g = 9.81 m.s-2) while surge and heave are both 0. When the front of the logger is vertically pointing down (forward acceleration, x), surge is +1g and when it is rolled to the right (sidewards acceleration, y) sway is +1g. Due to gravity, the pitch of a logger is thus translated into the values of heave, surge and sway (or z, x, y).

However, the values are measured in mV instead of g. Before delivering the GPS-loggers, they have been calibrated at UvA. The acceleration (A) expressed in g is calculated from the measured value (M) and the calibration factors offset (''O'') and sensitivity (''S'').

Thus acceleration for heave (z), surge (x) and sway (y) is calculated as:
Az = (Mz-Oz)/Sz ; Ax = (Mx-Ox)/Sx; Ay = (My-Oy)/Sy ;

The values of ''O'' (in general ranging from 0-60, sometimes even 300) and ''S'' (ranging from 1300-1450) are available in the device table of the data base. If not available, use the factory defaults ''O'' = 0 and ''S'' = 1365.

{|
| [[File:accelero1.png|300px]]||[[File:accelero2.png|300px]]|
|}

In the figure above heave (z) is shown in green, surge(x) in red and sway (y) in blue, all values are expressed in g (on the y-axis).

== Data processing and filtering ==
The raw log files are processed before being entered into the database. Each firmware version has a dedicated parser for the task. In certain cases data that is found in the log files is not stored in the database. This can occur for two reasons:

1. The data did not pass the CRC8 checksum (cyclic redundancy check). This is a standard procedure designed to detect accidental changes to raw computer data.

2. If the data has an invalid signature. For example, if a certain data format (eg numeric) or number of values (eg 4 digits) is expected but not returned, this record is not stored in the database.

When a line is deleted, the log file name, line number and the type of error is stored in the table gps.uva_warning102

== Information about GPS receivers ==
The first series of tags used the UBlox LEAS4S chip (~up to tag 541), newer tags use UBlox LEAS6S. The series developed in 2012 will use the Max6G receivers. From 2012 receiver versions will be stored in the database.
The UBX protocol is used for programming firmware, packaging and processing data.
For more information about the receivers themselves see the ublox documentation.
Ublox 4 Protocol specification [http://www.u-blox.com/images/downloads/Product_Docs/ANTARIS_Protocol_Specification%28GPS.G3-X-03002%29.zip]
Ublox 6 Protocol specification [http://www.u-blox.com/images/downloads/Product_Docs/u-blox6_ReceiverDescriptionProtocolSpec_%28GPS.G6-SW-10018%29.pdf]

UvA-BiTS Tracking Data

2014-12-11T10:22:02Z

Judy: /* Accelerometer calibration */

== Tracking data ==

'''Note:''' These views will show no rows for trackers not yet attached to birds.

=== ''gps.''ee_tracking_speed_limited ===

Shows the tracking data that your projects allow you to see.

Each row represents a GPS fix as it is interpreted from the data that comes via the GPS tracker, plus it adds speeds.

It replaces old views gps.uva_tracking_limited, gps.uva_tracking_speed_3d_limited and gps.uva_tracking_speed. This table structure has ''not'' changed with respect to our previous database.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker that reported the fix. Part of the unique identifier of the GPS fix.
|-
|date_time||timestamp||NO||Time when the fix was taken. UTC (i.e.: no timezone). Part of the unique identifier of the GPS fix.
|-
|latitude||float8||YES||Latitude of the GPS fix, decimal degrees.
|-
|longitude||float8||YES||Longitude of the GPS fix, decimal degrees.
|-
|altitude||int4||YES||Altitude above sea level measured by GPS tag in meters
|-
|pressure||int4||YES||Pressure measured by GPS tag sensor in Pascals
|-
|temperature||float8||YES||Temperature measured by GPS tag sensor in Celsius
|-
|h_accuracy||float8||YES||Horizontal accuracy (meters)
|-
|v_accuracy||float8||YES||vertical accuracy (meters)
|-
|x_speed||float8||YES||meters/second
|-
|y_speed||float8||YES||meters/second
|-
|z_speed||float8||YES||meters/second
|-
|gps_fixtime||float8||YES||GPS fix time (also known as Time to Fix) in seconds
|-
|userflag||int4||YES||Acceptable = 0; Data flagged as unacceptable by user if not equal to 0.
|-
|satellites_used||int2||YES||Number of satellites used for fix the GPS measurement
|-
|positiondop||float8||YES||Positional DOP (Dilution of Precision), unitless
|-
|speed_accuracy||float8||YES||
|-
|location||geometry||YES||postgreSQL geometry; to allow GIS operations
|-
|vnorth||float8||YES||- vx * sin(lat) * cos(long) - vy * sin(lat) * sin (long) + vz * cos(lat), m/s
|-
|veast||float8||YES||- vx * sin(lon) + vy * cos(lon), m/s
|-
|vdown||float8||YES||- vx * cos(lat) * cos(lon) - vy * cos(lat) * sin(lon) - vz * sin(lat), m/s
|-
|speed||float8||YES||- vx * sin(lon) + vy * cos(lon)
|-
|speed_3d||float8||YES|| sqrt(d.x_speed^2 + d.y_speed^2 + d.z_speed^2), m/s
|-
|speed_2d||float8||YES|| Speed on 2D, m/s
|-
|direction||numeric||YES|| Heading in degrees; 0 is north, 90 is east
|-
|altitude_agl||float8||YES|| Reported GPS altitude MINUS terrain elevation on the coordinate based on Digital Elevation Model data used in the e-ecology database (for further information see: [[Elevation]]
|}

=== ''gps.''ee_acceleration_limited ===

Each row represents an acceleration measurement as it is interpreted from the data that comes via the accelerometer. Acceleration is looked at in batches for a period of time, for 3 components (X,Y,Z). The blocks are delimited by a starting date and time, and then each observation is numbered consecutively.

The logger tries to mark the start of blocks from the GPS fix that coincides with the beginning of the acceleration block. If there's no GPS fix during that block, then it extrapolates the date and time and makes a special record which is interpreted by our accelerometer processing software and you can find it via gps.ee_acc_start_limited. Therefore, to know all valid dates and times that mark a valid block for a tracker, you need to consider all of the values in column date_time on gps.ee_tracking_speed_limited for the given tracker, column date_time on gps.ee_acc_start_limited, and all the values on column date_time for that tracker. And the order is important within each block, so you should sort the acceleration measurements by block and index.

This table structure has ''not'' changed with respect to our previous database.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker that reported the fix. Part of the unique identifier of the GPS fix.
|-
|date_time||timestamp||NO||Time of the acceleration block that this measurement belongs to.
|-
|index||int2||NO||Sorting order of the measurement within the acceleration block (each block starts at 0).
|-
|x_acceleration||int2||YES||Acceleration measured on the x (or surge) axis (range between -2666 and 2666).
|-
|y_acceleration||int2||YES||Acceleration measured on the y (or sway) axis (range between -2666 and 2666).
|-
|z_acceleration||int2||YES||Acceleration measured on the z (or heave) axis (range between -2666 and 2666).
|}
=== ''gps.''ee_acc_start_limited ===

Each row represents the start of a block of acceleration measurements when a GPS fix is not available at that acceleration block's time.

The logger tries to mark the start of blocks from the GPS fix that coincides with the begin of the acceleration block. If there's no GPS fix during that block, then it extrapolates the date and time and makes a special record which is interpreted by our accelerometer processing software and made available through this view.

Therefore, to know all valid dates and times that mark a valid block for a tracker, you need to consider all of the values in column date_time on gps.ee_tracking_speed_limited for the given tracker, column date_time on gps.ee_acc_start_limited, and all the values on column date_time for that tracker. And the order is important within each block, so you should sort the acceleration measurements by block and index.

Added in firmware 1.0.2.7.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker that reported the acceleration measurement.
|-
|date_time||timestamp||NO||Time to mark the start of the acceleration block.
|-
|line_counter||int4||NO||Number of lines associated with this set of accelerations.
|-
|timesynced||int2||YES||The source of the time that marks the acceleration block (0=not synced, 1=user set, 2=Gps Synced).
|-
|ascii||int4||YES||Acceleration measurement interval.
|-
|accsn||int4||YES||Number of samples (0-65536).
|-
|f||int2||YES||Acceleration sampling frequency (0=20Hz, 1=10Hz, 2=5Hz, 3=1Hz).
|}

== Bird data ==
=== ''gps.''ee_individual_limited ===

Each row shows bird's details for all birds that your projects allow you to see.

This view replaces the old gps.uva_individual_limited.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|ring_number||varchar||NO||Ring number.
|-
|species_latin_name||varchar||NO||Latin name of the species of the bird.
|-
|colour_ring||varchar||YES||String for ring colour.
|-
|sex||varchar||NO||Sex (M=male, F=female, X=unknown/unspecified)
|-
|mass||numeric(5,0)||YES||Mass (grams)
|-
|start_date||timestamp||NO||Date when the bird was first tracked with an UvA-BiTS logger.
|-
|end_date||timestamp||NO||Date at which the bird is no longer relevant for the tracking project.
|-
|remarks||text||YES||Text for user remarks
|-
|individual_id||int8||NO||Bird unique id. Included just to enable joins.
|}

=== ''gps.''ee_nest_limited ===
This view shows nests you have access to. A nest is always for a bird with a tracker (so a track session must exist for that bird), and must contain a short name (a reference name for you to use) and a latitude and longitude pair.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|reference_name||varchar||NO||A string with a useful reference for you. We recommend its being unique for each nest in a project, but the system does NOT enforce it.
|-
|latitude||numeric||YES||Latitude of the coordinates where the nest was found, decimal degrees.
|-
|longitude||numeric||YES||Longitude of the coordinates where the nest was found, decimal degrees.
|-
|start_date_time||timestamp||NO||When the nest was spotted.
|-
|end_date_time||timestamp||NO||When the nest ceases to be relevant.
|-
|found_by_whom||varchar||YES||Place to credit people involved in finding the nest.
|-
|remarks||text||YES||Free text.
|-
|location||bytearray||YES||Calculated Geometry of the coordinates where the nest was found.
|-
|nest_id||int8||NO||Unique id. Added to allow joins.
|}

=== ''gps.''ee_nest_inhabitant_limited ===
This view shows who lives on which nest, for birds you have access to.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|reference_name||varchar||NO||A string belonging to the nest that you can use to identify that nest. Because it is NOT enforced to be unique, DO NOT USE THIS FIELD FOR JOINING.
|-
|ring_number||varchar||NO||The ring number of a bird on a nest.
|-
|device_info_serial||int8||NO||Following the link from a bird to her track sessions, and considering the nest's start and end time, the bird's trackers within that period are resolved so that they can be shown here. This is the device_info_serial of a tracker.
|-
|key_name||varchar||NO||Following the link from a bird to her track sessions, and considering the nest's start and end time, the track sessions' projects within that period are resolved so that they can be shown here. This is the key_name of a project.
|-
|nest_id||int8||NO||Reference to the nest id. Added to allow joins.
|-
|individual_id||int8||NO||Reference to the individual id. Added to allow joins.
|-
|track_session_id||int8||NO||Reference to a resolved track session id. Added to allow joins.
|}
== Tracker and project data ==

=== ''gps.''ee_project_limited ===

Each row shows the details of each project, for the projects you are allowed to see.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||Unique key name, with the form <species>_<place>
|-
|station_name||varchar||NO||Where the receiving station is
|-
|start_date||timestamp||NO||Project start date
|-
|end_date||timestamp||NO||Date at which the project is not relevant any more
|-
|description||text||YES||String for free description. Updatable by the user.
|-
|project_id||int8||NO||Project unique id. Included just to enable joins.
|-
|parent_id||int8||YES||Reference to the parent project in the hierarchy.
|}

=== ''gps.''ee_tracker_limited ===

Each row shows tracker details for all trackers that your projects allow you to see.

This view replaces the old gps.uva_device_limited.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int8||NO||Serial number of the tracker
|-
|firmware_version||varchar||YES||String for firmware version
|-
|mass||numeric(4,2)||YES||Mass (in grams)
|-
|start_date||timestamp||NO||Date before which the device is not relevant
|-
|end_date||timestamp||NO||Date at which the device is not relevant any more
|-
|x_o||numeric(30,6)||YES||Accelerometer calibration: offset on the x (or surge) axis
|-
|x_s||numeric(30,6)||YES||Accelerometer calibration: sensitivity on the x (or surge) axis
|-
|y_o||numeric(30,6)||YES||Accelerometer calibration: offset on the y (or sway) axis
|-
|y_s||numeric(30,6)||YES||Accelerometer calibration: sensitivity on the y (or sway) axis
|-
|z_o||numeric(30,6)||YES||Accelerometer calibration: offset on the z (or heave) axis
|-
|z_s||numeric(30,6)||YES||Accelerometer calibration: sensitivity on the y (or heave) axis
|-
|tracker_id||int8||NO||Tracker unique id. Included just to enable joins.
|}

=== ''gps.''ee_track_session_limited ===

A track session links a tracker and a bird for a given period of time. Therefore, it is '''mandatory for projects''' to supply this piece of '''metadata''' before they can see any tracking data related to a tracker they own.

In order to create a track session, the project must create first an entry in the system for the bird that they are attaching the tracker to, and this is the way for the UvA-BiTS community to make sense of the tracking data that we are receiving. This is why providing the metadata is so important.

By editing the start and end dates, administrators of the project that this track session belongs to, can delimit (and therefore, establish) when the coupling makes sense (or is to be understood as useful).

Whenever a tracker is attached to a bird, a track session must be started (start_date) to indicate that the tracking data that is received is already live data. If the tracker is separated from the bird, the tracking data ceases to make sense, so the track session's end date should be set to denote it has ended. New track sessions can be started if the tracker is re-attached to a bird (possibly the same as before, or maybe a different one) afterwards.

Each row shows track session's information. There exists a row for each track session that your projects allow you to see.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||The project that the track session belongs to.
|-
|device_info_serial||int8||NO||Serial number of the tracker this track session involves.
|-
|ring_number||varchar||NO||Ring number of the bird that this track session involves.
|-
|start_date||timestamp||NO||Date before which the track session is not relevant
|-
|end_date||timestamp||NO||Date at which the track session is not relevant any more
|-
|remarks||text||YES||Free text for comments
|-
|start_latitude||numeric(11,8)||NO||Latitude where the bird was tagged, decimal degrees
|-
|start_longitude||numeric(11,8)||NO||Longitude where the bird was tagged, decimal degrees
|-
|project_id||int8||NO||Reference to the project this track session belongs to. Included just to enable joins.
|-
|tracker_id||int8||NO||Reference to the tracker this track session involves. Included just to enable joins.
|-
|individual_id||int8||NO||Reference to the individual this track session involves. Included just to enable joins.
|-
|track_session_id||int8||NO||Unique identifier of the track session. Included just to enable joins.
|}

== Relation to projects ==

=== ''gps.''ee_user_limited ===

Each row shows a colleague in a project that you belong to, indicating her role and status there.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||Project name
|-
|email||varchar||NO||e-mail address
|-
|login_name||varchar||NO||Login name
|-
|name||varchar||NO||(first) name
|-
|surname||varchar||NO||Surname
|-
|role||varchar||NO||Role in the project
|-
|status||varchar||NO||Status of the membership
|-
|user_id||int8||NO||Unique identifier of the user. Included to enable joins.
|}

=== ''gps.''ee_tracker_ownership_limited ===

Each row that you see expresses that a tracker belongs to one of the projects you are involved with.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||The project that the tracker belongs to.
|-
|device_info_serial||int8||NO||Serial number of the tracker.
|-
|project_id||int8||NO||Reference to the project. Included just to enable joins.
|-
|tracker_id||int8||NO||Reference to the tracker. Included just to enable joins.
|}

=== ''gps.''ee_individual_ownership_limited ===

Each row that you see expresses that a bird belongs to one of the projects you are involved with.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||The project that the individual belongs to.
|-
|ring_number||varchar||NO||Ring number of the bird.
|-
|project_id||int8||NO||Reference to the project. Included just to enable joins.
|-
|individual_id||int8||NO||Reference to the individual. Included just to enable joins.
|}

== Engineering data ==

'''Note:''' These views will show no rows for trackers not yet attached to birds.

=== ''gps.''ee_comm_limited ===

Each row indicates status and result of communications with ground stations for each tracker that your project allows you to see.

Only for newer firmware (v.1.0.1.4 and higher).

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
| device_info_serial || int4 || NO || Serial number of the tracker.
|-
| date_time || timestamp || NO || timestamp of start communication
|-
| date_time_end || timestamp ||YES|| timestamp of end communication
|-
| date_time_utc || timestamp ||YES|| date_time in utc (if available)
|-
| date_time_end_utc || timestamp ||YES|| date_time_end in utc (if available)
|-
| action || text||YES|| JOINED / RESCHEDULE ??
|-
| bytes_communicated || int4 ||YES|| Number of bytes communicated (total of all bytes per DATA line)
|-
|status || text ||YES|| READY, TIMEOUT, LOWPOWER, MISSING
|-
| network_join_time || int4||YES||milliseconds
|-
| number_identical_lines || int4 ||YES|| Number identical lines in data block. Gives indication
|-
| non_ascii_char || int4 ||YES|| Number of non ascii characters (indicated by question-mark ?)
|-
| sts_time || timestamp ||YES|| derived from S0 hh,mi
|-
| timesynced || int2||YES|| TimeSynced (0=not sy:wnced, 1=user set, 2=Gps Synced)
|-
|devicemode || int2||YES||DeviceMode (0-99). This describes whether the device is in a special operational mode. 0 = normal operational mode; 80 hex = Memory save mode (memory is almost full and all activity is drastically reduced so the circular memory probably won't be overwritten or, if it does, it will be kept to a minimum)
|-
| vbat || float8 ||YES|| Battery voltage in V (0-~4.200V)
|-
|logb || int4 ||YES|| logged bytes, waiting for download (0-4Mbyte)
|-
| logp || int2 ||YES|| Logged percentage (units 0.1%) of memory waiting to be downloaded (0-1000)
|-
| gpsii || int4 ||YES|| GpsIval, current (0-65535)
|-
|comii || int4||YES|| ComIval, current (0-65535)
|-
|accii || int4||YES|| AcceleroIval, current (0-65535)
|-
|swddd || int4||YES|| User switches, current (0-65535)
|-
|last_gps_fix ||timestamp||YES||timestamp of last fix
|-
|latitude||float8||YES||Latitude of last fix. Decimal degrees
|-
|longitude||float8||YES||Longitude of last fix. Decimal degrees
|}

=== ''gps.''ee_energy_limited ===

Only for newer firmware.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker
|-
|date_time||timestamp||NO||Date and time of the reported data
|-
|timesynced ||int2||YES||TimeSynced (0=not synced, 1=user set, 2=Gps synced)
|-
|devicemode ||int2||YES||(0-99)
|-
|vsll ||float8||YES||Solar Voltage during charging in mV (0-~2500)
|-
|vbat ||float8||YES||Battery Voltage in V (0-~4.200)
|-
|ssw ||int2||YES|| Solar Cell Switches ... (0-255) fw1022 and up
|-
| ||||YES||extended fields if special user switch is on
|-
|vsoo ||float8||YES|| Solar voltage open in V (0-~2.500)
|-
|vdrf ||float8||YES||reference voltage Dac in V (0-~2.435)
|-
|vchr ||float8||YES||charge current voltage
|-
|g ||int2||YES|| Gps battery threshold OK
|-
|c || int2||YES||Com battery threshold OK
|-
|m || int2||YES||Charge Mode (0,1,2 Com battery threshold OK
|-
|pressure || int4||YES|| in pascal (1mBar = 100Pa, 1pressuremeter = ~12Pa)
|-
|temperature || float8||YES|| in celcius degrees
|}

=== ''gps.''ee_settings_limited ===

Settings are either from C0-C1-C9 lines or parsed from uplink communication lines.
If C0-C1 is parsed its values are compared with the previous value and if the settings are the same, then no new records need to be added.
If uplink settings are parsed, then wait until the last setting has been seen (DL command or EOF); then store all the new settings in one record. Only for newer firmware.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||
|-
|date_time||timestamp||NO||Derived timestamp from Hh:mi fields in C0 combined with computer timestamp in JOINED line
|-
|timesynced || int2 ||YES||
|-
|timezone_time || int2 ||YES||
|-
|swddd || int4||YES|| SpecialUserSwitches
|-
|gpsnavmode || int2||YES|| GpsNavMode 0-8. This is the Kalman filter used for GPS navigation mode often called the "Dynamic platform model". The following codes are provided by the GPS chip manufacturer for the UBlox GPS chip LEAS4S: 1 = stationary, 2 = pedestrian, 3 = automotive, 4 = sea, 5 = airborne 1G, 6 = airborne 2G, 7 = airborne 3G. The UBlox GPS chip LEAS6S uses the following codes: 0 = portable, 2 = stationary, 3 = pedestrian, 4 = automotive, 5 = sea, 6 = airborne with < 1g acceleration, 7 = airborne with < 2g acceleration, 8 = airborne with < 4g acceleration
|-
|gpsreceivermode || int2 ||YES|| GpsReceiverNode 0-3
|-
|gpsfixtime || int2||YES|| GpsFixTime 0-255
|-
|gps_ivaln_on_h || int2||YES|| hours Gps IvalN on
|-
|gps_ivaln_on_m || int2||YES|| minutes Gps IvalN on
|-
|gps_ivaln || int4||YES|| GpsIvalN (0-65530)
|-
|gps_ivale_on_h || int2||YES|| hours Gps IvalE on
|-
|gps_ivale_on_m || int2||YES|| minutes GpS IvalE on
|-
|gps_ivale || int4||YES|| GpsIvalE
|-
|com_ivaln_on_h || int2||YES|| hours Com IvalN on
|-
|com_ivaln_on_m || int2||YES|| minutes Com IvalN on
|-
|com_ivaln || int4||YES|| ComIvalN (0-65530)
|-
|com_ivale_on_h || int2||YES|| hours Com IvalE on
|-
|com_ivale_on_m || int2||YES|| minutes Com IvalE on
|-
|com_ivale || int4||YES|| ComIvalE
|-
|accii || int4||YES|| Accelero measurement interval
|-
|accbn || int4||YES|| Accelero number of blocks
|-
|accsn || int4||YES|| Accelero number of samples (0-65536)
|-
|f || int2||YES||Accelero sample frequency (0=20Hz, 1=10Hz, 2=5Hz, 3=1Hz)
|-
|pdp || int2||YES|| PdopMsk (3-250)
|-
|pac||int2||YES|| PaccMsk (10-100)
|-
|tdp ||int2||YES|| TdopMsk (3-250)
|-
|tac ||int2||YES|| TaccMsk (30-300)
|-
|ga || int2||YES|| Additional Fixes GpsIval mode (3-25) before interval fix is logged
|-
|aol || int2||YES||AccMemOffLev (100 - 950) units 0.1%; if Memfull above this level all Accelero activity will be overruled and switched off untill MemFull < level - 5
|-
|eventnum || text||YES|| 12 eventnumbers (0-0xff) only printed if not zero, therfore converted to text
|}

=== ''gps.''ee_trackingfile_limited ===

Each row shows the status of the last attempt to parse (process) the raw data of a file for a given tracker, for each tracker that your projects allow you to see.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|id||int8||NO||Unique identifier of the row
|-
|date_reported||date||NO||Date that the file arrived
|-
|filename||varchar||NO||Name of the file
|-
|last_modified_date||timestamp||NO||Date that the file was last modified on
|-
|device_info_serial||int4||YES||Serial number of the tracker this file comes from
|-
|filesize||int8||NO||Size of the file in KB
|-
|parsed_date||timestamp||YES||Date and time that the file was last attempted to be parsed on
|-
|version||int4||NO||Optimistic locking field. Ignore it as a user.
|-
|last_found_date||timestamp||YES||Last time that a process that runs to look for new files saw the file
|-
|n_lines||int4||YES||Count of lines found in the file last time it was processed
|-
|n_data_lines||int4||YES||Count of actual data lines, out of the total lines
|-
|n_tracking_records||int4||YES||Count of GPS fixes found during the parsing process
|-
|n_communication_records||int4||YES||Count of communication records found during the parsing process
|-
|n_energy_records||int4||YES||Count of energy records found during the parsing process
|-
|n_acceleration_records||int4||YES||Count of acceleration records found during the parsing process
|-
|n_acceleration_start_records||int4||YES||Count of acceleration start records found during the parsing process
|-
|n_errors||int4||YES||Count of errors encountered during the parsing process
|-
|n_settings_records||int4||YES||Count of settings records found during the parsing process
|-
|filepath||text||NO||Directory path of the file on the server
|}

=== ''gps.''ee_warning_limited ===

Since firmware version 1.0.2.7 this table was added for events that do not occur very often (C10, C20) or errors that occur during parsing.

For example all CRC failures can be selected with :

select * from gps.ee_warning_limited
where warning = 'CRC8 failure'
order by device_info_serial;

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker
|-
|filename||text||NO||File name
|-
|linenumber||integer||YES||line number in file
|-
|warning||text||YES||The warning message
|-
|line||text||YES||The actual line contents in the file
|}

== Sharings ==

Projects can now share trackers with other projects during periods of time. This means that people in the receiving projects can access information that these shared trackers provide during those time periods.

The following views allow you to see this shared information:

* Tracking data
** ''gps.''ee_shared_tracking_speed_limited: tracking information (gps, speed), during the shared period, for trackers shared
** ''gps.''ee_shared_acc_start_limited: acceleration start information, during the shared period, for trackers shared
** ''gps.''ee_shared_acceleration_limited: acceleration information for trackers shared
* ''Meta'' data
** ''gps.''ee_shared_tracker_limited: information about each shared tracker
** ''gps.''ee_shared_individual_limited: information about birds with shared trackers
** ''gps.''ee_shared_track_session_limited: information about track sessions which involve shared trackers
* Relation to projects
** ''gps.''ee_shared_tracker_sharing_limited: view information for each sharing that your projects allow you to see

== Accelerometer calibration ==

The accelerometer measures the acceleration in 3D (20 times per second). Acceleration is caused by gravity (static component) and by a change in speed (dynamic component).
When the logger is static in horizontal position (solar cells up), gravity (green arrow) shows a heave (vertical acceleration, z) of +1 g (1 g = 9.81 m.s-2) while surge and heave are both 0. When the front of the logger is vertically pointing down (forward acceleration, x), surge is +1g and when it is rolled to the right (sidewards acceleration, y) sway is +1g. Due to gravity, the pitch of a logger is thus translated into the values of heave, surge and sway (or z, x, y).

However, the values are measured in mV instead of g. Before delivering the GPS-loggers, they have been calibrated at UvA. The acceleration (A) expressed in g is calculated from the measured value (M) and the calibration factors offset (''O'') and sensitivity (''S'').

Thus acceleration for heave (z), surge (x) and sway (y) is calculated as:
Az = (Mz-Oz)/Sz ; Ax = (Mx-Ox)/Sx; Ay = (My-Oy)/Sy ;

The values of ''O'' (in general ranging from 0-60, sometimes even 300) and ''S'' (ranging from 1300-1450) are available in the device table of the data base. If not available, use the factory defaults ''O'' = 0 and ''S'' = 1365.

{|
| [[File:accelero1.png|300px]]||[[File:accelero2.png|300px]]|
|}

In the figure above heave (z) is shown in green, surge(x) in red and sway (y) in blue, all values are expressed in g (on the y-axis).

== Data processing and filtering ==
The raw log files are processed before being entered into the database. Each firmware version has a dedicated parser for the task. In certain cases data that is found in the log files is not stored in the database. This can occur for two reasons:

1. The data did not pass the CRC8 checksum (cyclic redundancy check). This is a standard procedure designed to detect accidental changes to raw computer data.

2. If the data has an invalid signature. For example, if a certain data format (eg numeric) or number of values (eg 4 digits) is expected but not returned, this record is not stored in the database.

When a line is deleted, the log file name, line number and the type of error is stored in the table gps.uva_warning102

== Information about GPS receivers ==
The first series of tags used the UBlox LEAS4S chip (~up to tag 541), newer tags use UBlox LEAS6S. The series developed in 2012 will use the Max6G receivers. From 2012 receiver versions will be stored in the database.
The UBX protocol is used for programming firmware, packaging and processing data.
For more information about the receivers themselves see the ublox documentation.
Ublox 4 Protocol specification [http://www.u-blox.com/images/downloads/Product_Docs/ANTARIS_Protocol_Specification%28GPS.G3-X-03002%29.zip]
Ublox 6 Protocol specification [http://www.u-blox.com/images/downloads/Product_Docs/u-blox6_ReceiverDescriptionProtocolSpec_%28GPS.G6-SW-10018%29.pdf]

UvA-BiTS Tracking Data

2014-12-11T10:19:06Z

Judy: /* Meta data */

== Tracking data ==

'''Note:''' These views will show no rows for trackers not yet attached to birds.

=== ''gps.''ee_tracking_speed_limited ===

Shows the tracking data that your projects allow you to see.

Each row represents a GPS fix as it is interpreted from the data that comes via the GPS tracker, plus it adds speeds.

It replaces old views gps.uva_tracking_limited, gps.uva_tracking_speed_3d_limited and gps.uva_tracking_speed. This table structure has ''not'' changed with respect to our previous database.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker that reported the fix. Part of the unique identifier of the GPS fix.
|-
|date_time||timestamp||NO||Time when the fix was taken. UTC (i.e.: no timezone). Part of the unique identifier of the GPS fix.
|-
|latitude||float8||YES||Latitude of the GPS fix, decimal degrees.
|-
|longitude||float8||YES||Longitude of the GPS fix, decimal degrees.
|-
|altitude||int4||YES||Altitude above sea level measured by GPS tag in meters
|-
|pressure||int4||YES||Pressure measured by GPS tag sensor in Pascals
|-
|temperature||float8||YES||Temperature measured by GPS tag sensor in Celsius
|-
|h_accuracy||float8||YES||Horizontal accuracy (meters)
|-
|v_accuracy||float8||YES||vertical accuracy (meters)
|-
|x_speed||float8||YES||meters/second
|-
|y_speed||float8||YES||meters/second
|-
|z_speed||float8||YES||meters/second
|-
|gps_fixtime||float8||YES||GPS fix time (also known as Time to Fix) in seconds
|-
|userflag||int4||YES||Acceptable = 0; Data flagged as unacceptable by user if not equal to 0.
|-
|satellites_used||int2||YES||Number of satellites used for fix the GPS measurement
|-
|positiondop||float8||YES||Positional DOP (Dilution of Precision), unitless
|-
|speed_accuracy||float8||YES||
|-
|location||geometry||YES||postgreSQL geometry; to allow GIS operations
|-
|vnorth||float8||YES||- vx * sin(lat) * cos(long) - vy * sin(lat) * sin (long) + vz * cos(lat), m/s
|-
|veast||float8||YES||- vx * sin(lon) + vy * cos(lon), m/s
|-
|vdown||float8||YES||- vx * cos(lat) * cos(lon) - vy * cos(lat) * sin(lon) - vz * sin(lat), m/s
|-
|speed||float8||YES||- vx * sin(lon) + vy * cos(lon)
|-
|speed_3d||float8||YES|| sqrt(d.x_speed^2 + d.y_speed^2 + d.z_speed^2), m/s
|-
|speed_2d||float8||YES|| Speed on 2D, m/s
|-
|direction||numeric||YES|| Heading in degrees; 0 is north, 90 is east
|-
|altitude_agl||float8||YES|| Reported GPS altitude MINUS terrain elevation on the coordinate based on Digital Elevation Model data used in the e-ecology database (for further information see: [[Elevation]]
|}

=== ''gps.''ee_acceleration_limited ===

Each row represents an acceleration measurement as it is interpreted from the data that comes via the accelerometer. Acceleration is looked at in batches for a period of time, for 3 components (X,Y,Z). The blocks are delimited by a starting date and time, and then each observation is numbered consecutively.

The logger tries to mark the start of blocks from the GPS fix that coincides with the beginning of the acceleration block. If there's no GPS fix during that block, then it extrapolates the date and time and makes a special record which is interpreted by our accelerometer processing software and you can find it via gps.ee_acc_start_limited. Therefore, to know all valid dates and times that mark a valid block for a tracker, you need to consider all of the values in column date_time on gps.ee_tracking_speed_limited for the given tracker, column date_time on gps.ee_acc_start_limited, and all the values on column date_time for that tracker. And the order is important within each block, so you should sort the acceleration measurements by block and index.

This table structure has ''not'' changed with respect to our previous database.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker that reported the fix. Part of the unique identifier of the GPS fix.
|-
|date_time||timestamp||NO||Time of the acceleration block that this measurement belongs to.
|-
|index||int2||NO||Sorting order of the measurement within the acceleration block (each block starts at 0).
|-
|x_acceleration||int2||YES||Acceleration measured on the x (or surge) axis (range between -2666 and 2666).
|-
|y_acceleration||int2||YES||Acceleration measured on the y (or sway) axis (range between -2666 and 2666).
|-
|z_acceleration||int2||YES||Acceleration measured on the z (or heave) axis (range between -2666 and 2666).
|}
=== ''gps.''ee_acc_start_limited ===

Each row represents the start of a block of acceleration measurements when a GPS fix is not available at that acceleration block's time.

The logger tries to mark the start of blocks from the GPS fix that coincides with the begin of the acceleration block. If there's no GPS fix during that block, then it extrapolates the date and time and makes a special record which is interpreted by our accelerometer processing software and made available through this view.

Therefore, to know all valid dates and times that mark a valid block for a tracker, you need to consider all of the values in column date_time on gps.ee_tracking_speed_limited for the given tracker, column date_time on gps.ee_acc_start_limited, and all the values on column date_time for that tracker. And the order is important within each block, so you should sort the acceleration measurements by block and index.

Added in firmware 1.0.2.7.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker that reported the acceleration measurement.
|-
|date_time||timestamp||NO||Time to mark the start of the acceleration block.
|-
|line_counter||int4||NO||Number of lines associated with this set of accelerations.
|-
|timesynced||int2||YES||The source of the time that marks the acceleration block (0=not synced, 1=user set, 2=Gps Synced).
|-
|ascii||int4||YES||Acceleration measurement interval.
|-
|accsn||int4||YES||Number of samples (0-65536).
|-
|f||int2||YES||Acceleration sampling frequency (0=20Hz, 1=10Hz, 2=5Hz, 3=1Hz).
|}

== Bird data ==
=== ''gps.''ee_individual_limited ===

Each row shows bird's details for all birds that your projects allow you to see.

This view replaces the old gps.uva_individual_limited.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|ring_number||varchar||NO||Ring number.
|-
|species_latin_name||varchar||NO||Latin name of the species of the bird.
|-
|colour_ring||varchar||YES||String for ring colour.
|-
|sex||varchar||NO||Sex (M=male, F=female, X=unknown/unspecified)
|-
|mass||numeric(5,0)||YES||Mass (grams)
|-
|start_date||timestamp||NO||Date when the bird was first tracked with an UvA-BiTS logger.
|-
|end_date||timestamp||NO||Date at which the bird is no longer relevant for the tracking project.
|-
|remarks||text||YES||Text for user remarks
|-
|individual_id||int8||NO||Bird unique id. Included just to enable joins.
|}

=== ''gps.''ee_nest_limited ===
This view shows nests you have access to. A nest is always for a bird with a tracker (so a track session must exist for that bird), and must contain a short name (a reference name for you to use) and a latitude and longitude pair.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|reference_name||varchar||NO||A string with a useful reference for you. We recommend its being unique for each nest in a project, but the system does NOT enforce it.
|-
|latitude||numeric||YES||Latitude of the coordinates where the nest was found, decimal degrees.
|-
|longitude||numeric||YES||Longitude of the coordinates where the nest was found, decimal degrees.
|-
|start_date_time||timestamp||NO||When the nest was spotted.
|-
|end_date_time||timestamp||NO||When the nest ceases to be relevant.
|-
|found_by_whom||varchar||YES||Place to credit people involved in finding the nest.
|-
|remarks||text||YES||Free text.
|-
|location||bytearray||YES||Calculated Geometry of the coordinates where the nest was found.
|-
|nest_id||int8||NO||Unique id. Added to allow joins.
|}

=== ''gps.''ee_nest_inhabitant_limited ===
This view shows who lives on which nest, for birds you have access to.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|reference_name||varchar||NO||A string belonging to the nest that you can use to identify that nest. Because it is NOT enforced to be unique, DO NOT USE THIS FIELD FOR JOINING.
|-
|ring_number||varchar||NO||The ring number of a bird on a nest.
|-
|device_info_serial||int8||NO||Following the link from a bird to her track sessions, and considering the nest's start and end time, the bird's trackers within that period are resolved so that they can be shown here. This is the device_info_serial of a tracker.
|-
|key_name||varchar||NO||Following the link from a bird to her track sessions, and considering the nest's start and end time, the track sessions' projects within that period are resolved so that they can be shown here. This is the key_name of a project.
|-
|nest_id||int8||NO||Reference to the nest id. Added to allow joins.
|-
|individual_id||int8||NO||Reference to the individual id. Added to allow joins.
|-
|track_session_id||int8||NO||Reference to a resolved track session id. Added to allow joins.
|}
== Tracker and project data ==

=== ''gps.''ee_project_limited ===

Each row shows the details of each project, for the projects you are allowed to see.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||Unique key name, with the form <species>_<place>
|-
|station_name||varchar||NO||Where the receiving station is
|-
|start_date||timestamp||NO||Project start date
|-
|end_date||timestamp||NO||Date at which the project is not relevant any more
|-
|description||text||YES||String for free description. Updatable by the user.
|-
|project_id||int8||NO||Project unique id. Included just to enable joins.
|-
|parent_id||int8||YES||Reference to the parent project in the hierarchy.
|}

=== ''gps.''ee_tracker_limited ===

Each row shows tracker details for all trackers that your projects allow you to see.

This view replaces the old gps.uva_device_limited.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int8||NO||Serial number of the tracker
|-
|firmware_version||varchar||YES||String for firmware version
|-
|mass||numeric(4,2)||YES||Mass (in grams)
|-
|start_date||timestamp||NO||Date before which the device is not relevant
|-
|end_date||timestamp||NO||Date at which the device is not relevant any more
|-
|x_o||numeric(30,6)||YES||Accelerometer calibration: offset on the x (or surge) axis
|-
|x_s||numeric(30,6)||YES||Accelerometer calibration: sensitivity on the x (or surge) axis
|-
|y_o||numeric(30,6)||YES||Accelerometer calibration: offset on the y (or sway) axis
|-
|y_s||numeric(30,6)||YES||Accelerometer calibration: sensitivity on the y (or sway) axis
|-
|z_o||numeric(30,6)||YES||Accelerometer calibration: offset on the z (or heave) axis
|-
|z_s||numeric(30,6)||YES||Accelerometer calibration: sensitivity on the y (or heave) axis
|-
|tracker_id||int8||NO||Tracker unique id. Included just to enable joins.
|}

=== ''gps.''ee_track_session_limited ===

A track session links a tracker and a bird for a given period of time. Therefore, it is '''mandatory for projects''' to supply this piece of '''metadata''' before they can see any tracking data related to a tracker they own.

In order to create a track session, the project must create first an entry in the system for the bird that they are attaching the tracker to, and this is the way for the UvA-BiTS community to make sense of the tracking data that we are receiving. This is why providing the metadata is so important.

By editing the start and end dates, administrators of the project that this track session belongs to, can delimit (and therefore, establish) when the coupling makes sense (or is to be understood as useful).

Whenever a tracker is attached to a bird, a track session must be started (start_date) to indicate that the tracking data that is received is already live data. If the tracker is separated from the bird, the tracking data ceases to make sense, so the track session's end date should be set to denote it has ended. New track sessions can be started if the tracker is re-attached to a bird (possibly the same as before, or maybe a different one) afterwards.

Each row shows track session's information. There exists a row for each track session that your projects allow you to see.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||The project that the track session belongs to.
|-
|device_info_serial||int8||NO||Serial number of the tracker this track session involves.
|-
|ring_number||varchar||NO||Ring number of the bird that this track session involves.
|-
|start_date||timestamp||NO||Date before which the track session is not relevant
|-
|end_date||timestamp||NO||Date at which the track session is not relevant any more
|-
|remarks||text||YES||Free text for comments
|-
|start_latitude||numeric(11,8)||NO||Latitude where the bird was tagged, decimal degrees
|-
|start_longitude||numeric(11,8)||NO||Longitude where the bird was tagged, decimal degrees
|-
|project_id||int8||NO||Reference to the project this track session belongs to. Included just to enable joins.
|-
|tracker_id||int8||NO||Reference to the tracker this track session involves. Included just to enable joins.
|-
|individual_id||int8||NO||Reference to the individual this track session involves. Included just to enable joins.
|-
|track_session_id||int8||NO||Unique identifier of the track session. Included just to enable joins.
|}

== Relation to projects ==

=== ''gps.''ee_user_limited ===

Each row shows a colleague in a project that you belong to, indicating her role and status there.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||Project name
|-
|email||varchar||NO||e-mail address
|-
|login_name||varchar||NO||Login name
|-
|name||varchar||NO||(first) name
|-
|surname||varchar||NO||Surname
|-
|role||varchar||NO||Role in the project
|-
|status||varchar||NO||Status of the membership
|-
|user_id||int8||NO||Unique identifier of the user. Included to enable joins.
|}

=== ''gps.''ee_tracker_ownership_limited ===

Each row that you see expresses that a tracker belongs to one of the projects you are involved with.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||The project that the tracker belongs to.
|-
|device_info_serial||int8||NO||Serial number of the tracker.
|-
|project_id||int8||NO||Reference to the project. Included just to enable joins.
|-
|tracker_id||int8||NO||Reference to the tracker. Included just to enable joins.
|}

=== ''gps.''ee_individual_ownership_limited ===

Each row that you see expresses that a bird belongs to one of the projects you are involved with.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||The project that the individual belongs to.
|-
|ring_number||varchar||NO||Ring number of the bird.
|-
|project_id||int8||NO||Reference to the project. Included just to enable joins.
|-
|individual_id||int8||NO||Reference to the individual. Included just to enable joins.
|}

== Engineering data ==

'''Note:''' These views will show no rows for trackers not yet attached to birds.

=== ''gps.''ee_comm_limited ===

Each row indicates status and result of communications with ground stations for each tracker that your project allows you to see.

Only for newer firmware (v.1.0.1.4 and higher).

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
| device_info_serial || int4 || NO || Serial number of the tracker.
|-
| date_time || timestamp || NO || timestamp of start communication
|-
| date_time_end || timestamp ||YES|| timestamp of end communication
|-
| date_time_utc || timestamp ||YES|| date_time in utc (if available)
|-
| date_time_end_utc || timestamp ||YES|| date_time_end in utc (if available)
|-
| action || text||YES|| JOINED / RESCHEDULE ??
|-
| bytes_communicated || int4 ||YES|| Number of bytes communicated (total of all bytes per DATA line)
|-
|status || text ||YES|| READY, TIMEOUT, LOWPOWER, MISSING
|-
| network_join_time || int4||YES||milliseconds
|-
| number_identical_lines || int4 ||YES|| Number identical lines in data block. Gives indication
|-
| non_ascii_char || int4 ||YES|| Number of non ascii characters (indicated by question-mark ?)
|-
| sts_time || timestamp ||YES|| derived from S0 hh,mi
|-
| timesynced || int2||YES|| TimeSynced (0=not sy:wnced, 1=user set, 2=Gps Synced)
|-
|devicemode || int2||YES||DeviceMode (0-99). This describes whether the device is in a special operational mode. 0 = normal operational mode; 80 hex = Memory save mode (memory is almost full and all activity is drastically reduced so the circular memory probably won't be overwritten or, if it does, it will be kept to a minimum)
|-
| vbat || float8 ||YES|| Battery voltage in V (0-~4.200V)
|-
|logb || int4 ||YES|| logged bytes, waiting for download (0-4Mbyte)
|-
| logp || int2 ||YES|| Logged percentage (units 0.1%) of memory waiting to be downloaded (0-1000)
|-
| gpsii || int4 ||YES|| GpsIval, current (0-65535)
|-
|comii || int4||YES|| ComIval, current (0-65535)
|-
|accii || int4||YES|| AcceleroIval, current (0-65535)
|-
|swddd || int4||YES|| User switches, current (0-65535)
|-
|last_gps_fix ||timestamp||YES||timestamp of last fix
|-
|latitude||float8||YES||Latitude of last fix. Decimal degrees
|-
|longitude||float8||YES||Longitude of last fix. Decimal degrees
|}

=== ''gps.''ee_energy_limited ===

Only for newer firmware.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker
|-
|date_time||timestamp||NO||Date and time of the reported data
|-
|timesynced ||int2||YES||TimeSynced (0=not synced, 1=user set, 2=Gps synced)
|-
|devicemode ||int2||YES||(0-99)
|-
|vsll ||float8||YES||Solar Voltage during charging in mV (0-~2500)
|-
|vbat ||float8||YES||Battery Voltage in V (0-~4.200)
|-
|ssw ||int2||YES|| Solar Cell Switches ... (0-255) fw1022 and up
|-
| ||||YES||extended fields if special user switch is on
|-
|vsoo ||float8||YES|| Solar voltage open in V (0-~2.500)
|-
|vdrf ||float8||YES||reference voltage Dac in V (0-~2.435)
|-
|vchr ||float8||YES||charge current voltage
|-
|g ||int2||YES|| Gps battery threshold OK
|-
|c || int2||YES||Com battery threshold OK
|-
|m || int2||YES||Charge Mode (0,1,2 Com battery threshold OK
|-
|pressure || int4||YES|| in pascal (1mBar = 100Pa, 1pressuremeter = ~12Pa)
|-
|temperature || float8||YES|| in celcius degrees
|}

=== ''gps.''ee_settings_limited ===

Settings are either from C0-C1-C9 lines or parsed from uplink communication lines.
If C0-C1 is parsed its values are compared with the previous value and if the settings are the same, then no new records need to be added.
If uplink settings are parsed, then wait until the last setting has been seen (DL command or EOF); then store all the new settings in one record. Only for newer firmware.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||
|-
|date_time||timestamp||NO||Derived timestamp from Hh:mi fields in C0 combined with computer timestamp in JOINED line
|-
|timesynced || int2 ||YES||
|-
|timezone_time || int2 ||YES||
|-
|swddd || int4||YES|| SpecialUserSwitches
|-
|gpsnavmode || int2||YES|| GpsNavMode 0-8. This is the Kalman filter used for GPS navigation mode often called the "Dynamic platform model". The following codes are provided by the GPS chip manufacturer for the UBlox GPS chip LEAS4S: 1 = stationary, 2 = pedestrian, 3 = automotive, 4 = sea, 5 = airborne 1G, 6 = airborne 2G, 7 = airborne 3G. The UBlox GPS chip LEAS6S uses the following codes: 0 = portable, 2 = stationary, 3 = pedestrian, 4 = automotive, 5 = sea, 6 = airborne with < 1g acceleration, 7 = airborne with < 2g acceleration, 8 = airborne with < 4g acceleration
|-
|gpsreceivermode || int2 ||YES|| GpsReceiverNode 0-3
|-
|gpsfixtime || int2||YES|| GpsFixTime 0-255
|-
|gps_ivaln_on_h || int2||YES|| hours Gps IvalN on
|-
|gps_ivaln_on_m || int2||YES|| minutes Gps IvalN on
|-
|gps_ivaln || int4||YES|| GpsIvalN (0-65530)
|-
|gps_ivale_on_h || int2||YES|| hours Gps IvalE on
|-
|gps_ivale_on_m || int2||YES|| minutes GpS IvalE on
|-
|gps_ivale || int4||YES|| GpsIvalE
|-
|com_ivaln_on_h || int2||YES|| hours Com IvalN on
|-
|com_ivaln_on_m || int2||YES|| minutes Com IvalN on
|-
|com_ivaln || int4||YES|| ComIvalN (0-65530)
|-
|com_ivale_on_h || int2||YES|| hours Com IvalE on
|-
|com_ivale_on_m || int2||YES|| minutes Com IvalE on
|-
|com_ivale || int4||YES|| ComIvalE
|-
|accii || int4||YES|| Accelero measurement interval
|-
|accbn || int4||YES|| Accelero number of blocks
|-
|accsn || int4||YES|| Accelero number of samples (0-65536)
|-
|f || int2||YES||Accelero sample frequency (0=20Hz, 1=10Hz, 2=5Hz, 3=1Hz)
|-
|pdp || int2||YES|| PdopMsk (3-250)
|-
|pac||int2||YES|| PaccMsk (10-100)
|-
|tdp ||int2||YES|| TdopMsk (3-250)
|-
|tac ||int2||YES|| TaccMsk (30-300)
|-
|ga || int2||YES|| Additional Fixes GpsIval mode (3-25) before interval fix is logged
|-
|aol || int2||YES||AccMemOffLev (100 - 950) units 0.1%; if Memfull above this level all Accelero activity will be overruled and switched off untill MemFull < level - 5
|-
|eventnum || text||YES|| 12 eventnumbers (0-0xff) only printed if not zero, therfore converted to text
|}

=== ''gps.''ee_trackingfile_limited ===

Each row shows the status of the last attempt to parse (process) the raw data of a file for a given tracker, for each tracker that your projects allow you to see.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|id||int8||NO||Unique identifier of the row
|-
|date_reported||date||NO||Date that the file arrived
|-
|filename||varchar||NO||Name of the file
|-
|last_modified_date||timestamp||NO||Date that the file was last modified on
|-
|device_info_serial||int4||YES||Serial number of the tracker this file comes from
|-
|filesize||int8||NO||Size of the file in KB
|-
|parsed_date||timestamp||YES||Date and time that the file was last attempted to be parsed on
|-
|version||int4||NO||Optimistic locking field. Ignore it as a user.
|-
|last_found_date||timestamp||YES||Last time that a process that runs to look for new files saw the file
|-
|n_lines||int4||YES||Count of lines found in the file last time it was processed
|-
|n_data_lines||int4||YES||Count of actual data lines, out of the total lines
|-
|n_tracking_records||int4||YES||Count of GPS fixes found during the parsing process
|-
|n_communication_records||int4||YES||Count of communication records found during the parsing process
|-
|n_energy_records||int4||YES||Count of energy records found during the parsing process
|-
|n_acceleration_records||int4||YES||Count of acceleration records found during the parsing process
|-
|n_acceleration_start_records||int4||YES||Count of acceleration start records found during the parsing process
|-
|n_errors||int4||YES||Count of errors encountered during the parsing process
|-
|n_settings_records||int4||YES||Count of settings records found during the parsing process
|-
|filepath||text||NO||Directory path of the file on the server
|}

=== ''gps.''ee_warning_limited ===

Since firmware version 1.0.2.7 this table was added for events that do not occur very often (C10, C20) or errors that occur during parsing.

For example all CRC failures can be selected with :

select * from gps.ee_warning_limited
where warning = 'CRC8 failure'
order by device_info_serial;

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker
|-
|filename||text||NO||File name
|-
|linenumber||integer||YES||line number in file
|-
|warning||text||YES||The warning message
|-
|line||text||YES||The actual line contents in the file
|}

== Sharings ==

Projects can now share trackers with other projects during periods of time. This means that people in the receiving projects can access information that these shared trackers provide during those time periods.

The following views allow you to see this shared information:

* Tracking data
** ''gps.''ee_shared_tracking_speed_limited: tracking information (gps, speed), during the shared period, for trackers shared
** ''gps.''ee_shared_acc_start_limited: acceleration start information, during the shared period, for trackers shared
** ''gps.''ee_shared_acceleration_limited: acceleration information for trackers shared
* ''Meta'' data
** ''gps.''ee_shared_tracker_limited: information about each shared tracker
** ''gps.''ee_shared_individual_limited: information about birds with shared trackers
** ''gps.''ee_shared_track_session_limited: information about track sessions which involve shared trackers
* Relation to projects
** ''gps.''ee_shared_tracker_sharing_limited: view information for each sharing that your projects allow you to see

== Accelerometer calibration ==

The accelerometer measures the acceleration in 3D (20 times per second). Acceleration is caused by gravity (static component) and by a change in speed (dynamic component).
When the logger is static in horizontal position (solar cells up), gravity (green arrow) shows a heave (vertical acceleration, z) of +1 g (1 g = 9.81 m.s-2) while surge and heave are both 0. When the front of the logger is vertically pointing down (forward acceleration, x), surge is +1g and when it is rolled to the right (sidewards acceleration, y) sway is +1g. Due to gravity, the pitch of a logger is thus translated into the values of heave, surge and sway (or z, x, y).

However, the values are measured in mV instead of g. Before delivering the GPS-loggers, they have been calibrated at UvA. The acceleration (A) expressed in g is calculated from the measured value (M) and the calibration factors offset (''O'') and sensitivity (''S'').

Thus acceleration for heave (z), surge (x) and sway (y) is calculated as:
Az = (Mz-Oz)/Sz ; Ax = (Mx-Ox)/Sx; Ay = (My-Oy)/Sy ;

The values of ''O'' (in general ranging from 0-60, sometimes even 300) and ''S'' (ranging from 1300-1450) are available in the device table of the data base. If not available, use the factory defaults ''O'' = 0 and ''S'' = 1365.

{|
| [[File:accelero1.png|300px]]||[[File:accelero2.png|300px]]|
|}

In the figure above heave (z) is shown in green, surge(x) in red and sway (y) in blue, all values are expressed in g (on the y-axis).

UvA-BiTS Tracking Data

2014-12-11T10:12:21Z

Judy: /* Sharings */

== Tracking data ==

'''Note:''' These views will show no rows for trackers not yet attached to birds.

=== ''gps.''ee_tracking_speed_limited ===

Shows the tracking data that your projects allow you to see.

Each row represents a GPS fix as it is interpreted from the data that comes via the GPS tracker, plus it adds speeds.

It replaces old views gps.uva_tracking_limited, gps.uva_tracking_speed_3d_limited and gps.uva_tracking_speed. This table structure has ''not'' changed with respect to our previous database.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker that reported the fix. Part of the unique identifier of the GPS fix.
|-
|date_time||timestamp||NO||Time when the fix was taken. UTC (i.e.: no timezone). Part of the unique identifier of the GPS fix.
|-
|latitude||float8||YES||Latitude of the GPS fix, decimal degrees.
|-
|longitude||float8||YES||Longitude of the GPS fix, decimal degrees.
|-
|altitude||int4||YES||Altitude above sea level measured by GPS tag in meters
|-
|pressure||int4||YES||Pressure measured by GPS tag sensor in Pascals
|-
|temperature||float8||YES||Temperature measured by GPS tag sensor in Celsius
|-
|h_accuracy||float8||YES||Horizontal accuracy (meters)
|-
|v_accuracy||float8||YES||vertical accuracy (meters)
|-
|x_speed||float8||YES||meters/second
|-
|y_speed||float8||YES||meters/second
|-
|z_speed||float8||YES||meters/second
|-
|gps_fixtime||float8||YES||GPS fix time (also known as Time to Fix) in seconds
|-
|userflag||int4||YES||Acceptable = 0; Data flagged as unacceptable by user if not equal to 0.
|-
|satellites_used||int2||YES||Number of satellites used for fix the GPS measurement
|-
|positiondop||float8||YES||Positional DOP (Dilution of Precision), unitless
|-
|speed_accuracy||float8||YES||
|-
|location||geometry||YES||postgreSQL geometry; to allow GIS operations
|-
|vnorth||float8||YES||- vx * sin(lat) * cos(long) - vy * sin(lat) * sin (long) + vz * cos(lat), m/s
|-
|veast||float8||YES||- vx * sin(lon) + vy * cos(lon), m/s
|-
|vdown||float8||YES||- vx * cos(lat) * cos(lon) - vy * cos(lat) * sin(lon) - vz * sin(lat), m/s
|-
|speed||float8||YES||- vx * sin(lon) + vy * cos(lon)
|-
|speed_3d||float8||YES|| sqrt(d.x_speed^2 + d.y_speed^2 + d.z_speed^2), m/s
|-
|speed_2d||float8||YES|| Speed on 2D, m/s
|-
|direction||numeric||YES|| Heading in degrees; 0 is north, 90 is east
|-
|altitude_agl||float8||YES|| Reported GPS altitude MINUS terrain elevation on the coordinate based on Digital Elevation Model data used in the e-ecology database (for further information see: [[Elevation]]
|}

=== ''gps.''ee_acceleration_limited ===

Each row represents an acceleration measurement as it is interpreted from the data that comes via the accelerometer. Acceleration is looked at in batches for a period of time, for 3 components (X,Y,Z). The blocks are delimited by a starting date and time, and then each observation is numbered consecutively.

The logger tries to mark the start of blocks from the GPS fix that coincides with the beginning of the acceleration block. If there's no GPS fix during that block, then it extrapolates the date and time and makes a special record which is interpreted by our accelerometer processing software and you can find it via gps.ee_acc_start_limited. Therefore, to know all valid dates and times that mark a valid block for a tracker, you need to consider all of the values in column date_time on gps.ee_tracking_speed_limited for the given tracker, column date_time on gps.ee_acc_start_limited, and all the values on column date_time for that tracker. And the order is important within each block, so you should sort the acceleration measurements by block and index.

This table structure has ''not'' changed with respect to our previous database.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker that reported the fix. Part of the unique identifier of the GPS fix.
|-
|date_time||timestamp||NO||Time of the acceleration block that this measurement belongs to.
|-
|index||int2||NO||Sorting order of the measurement within the acceleration block (each block starts at 0).
|-
|x_acceleration||int2||YES||Acceleration measured on the x (or surge) axis (range between -2666 and 2666).
|-
|y_acceleration||int2||YES||Acceleration measured on the y (or sway) axis (range between -2666 and 2666).
|-
|z_acceleration||int2||YES||Acceleration measured on the z (or heave) axis (range between -2666 and 2666).
|}
=== ''gps.''ee_acc_start_limited ===

Each row represents the start of a block of acceleration measurements when a GPS fix is not available at that acceleration block's time.

The logger tries to mark the start of blocks from the GPS fix that coincides with the begin of the acceleration block. If there's no GPS fix during that block, then it extrapolates the date and time and makes a special record which is interpreted by our accelerometer processing software and made available through this view.

Therefore, to know all valid dates and times that mark a valid block for a tracker, you need to consider all of the values in column date_time on gps.ee_tracking_speed_limited for the given tracker, column date_time on gps.ee_acc_start_limited, and all the values on column date_time for that tracker. And the order is important within each block, so you should sort the acceleration measurements by block and index.

Added in firmware 1.0.2.7.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker that reported the acceleration measurement.
|-
|date_time||timestamp||NO||Time to mark the start of the acceleration block.
|-
|line_counter||int4||NO||Number of lines associated with this set of accelerations.
|-
|timesynced||int2||YES||The source of the time that marks the acceleration block (0=not synced, 1=user set, 2=Gps Synced).
|-
|ascii||int4||YES||Acceleration measurement interval.
|-
|accsn||int4||YES||Number of samples (0-65536).
|-
|f||int2||YES||Acceleration sampling frequency (0=20Hz, 1=10Hz, 2=5Hz, 3=1Hz).
|}

== ''Meta'' data ==

=== ''gps.''ee_project_limited ===

Each row shows the details of each project, for the projects you are allowed to see.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||Unique key name, with the form <species>_<place>
|-
|station_name||varchar||NO||Where the receiving station is
|-
|start_date||timestamp||NO||Project start date
|-
|end_date||timestamp||NO||Date at which the project is not relevant any more
|-
|description||text||YES||String for free description. Updatable by the user.
|-
|project_id||int8||NO||Project unique id. Included just to enable joins.
|-
|parent_id||int8||YES||Reference to the parent project in the hierarchy.
|}

=== ''gps.''ee_tracker_limited ===

Each row shows tracker details for all trackers that your projects allow you to see.

This view replaces the old gps.uva_device_limited.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int8||NO||Serial number of the tracker
|-
|firmware_version||varchar||YES||String for firmware version
|-
|mass||numeric(4,2)||YES||Mass (in grams)
|-
|start_date||timestamp||NO||Date before which the device is not relevant
|-
|end_date||timestamp||NO||Date at which the device is not relevant any more
|-
|x_o||numeric(30,6)||YES||Accelerometer calibration: offset on the x (or surge) axis
|-
|x_s||numeric(30,6)||YES||Accelerometer calibration: sensitivity on the x (or surge) axis
|-
|y_o||numeric(30,6)||YES||Accelerometer calibration: offset on the y (or sway) axis
|-
|y_s||numeric(30,6)||YES||Accelerometer calibration: sensitivity on the y (or sway) axis
|-
|z_o||numeric(30,6)||YES||Accelerometer calibration: offset on the z (or heave) axis
|-
|z_s||numeric(30,6)||YES||Accelerometer calibration: sensitivity on the y (or heave) axis
|-
|tracker_id||int8||NO||Tracker unique id. Included just to enable joins.
|}

=== ''gps.''ee_individual_limited ===

Each row shows bird's details for all birds that your projects allow you to see.

This view replaces the old gps.uva_individual_limited.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|ring_number||varchar||NO||Ring number.
|-
|species_latin_name||varchar||NO||Latin name of the species of the bird.
|-
|colour_ring||varchar||YES||String for ring colour.
|-
|sex||varchar||NO||Sex (M=male, F=female, X=unknown/unspecified)
|-
|mass||numeric(5,0)||YES||Mass (grams)
|-
|start_date||timestamp||NO||Date when the bird was first tracked with an UvA-BiTS logger.
|-
|end_date||timestamp||NO||Date at which the bird is no longer relevant for the tracking project.
|-
|remarks||text||YES||Text for user remarks
|-
|individual_id||int8||NO||Bird unique id. Included just to enable joins.
|}

=== ''gps.''ee_nest_limited ===
This view shows nests you have access to. A nest is always for a bird with a tracker (so a track session must exist for that bird), and must contain a short name (a reference name for you to use) and a latitude and longitude pair.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|reference_name||varchar||NO||A string with a useful reference for you. We recommend its being unique for each nest in a project, but the system does NOT enforce it.
|-
|latitude||numeric||YES||Latitude of the coordinates where the nest was found, decimal degrees.
|-
|longitude||numeric||YES||Longitude of the coordinates where the nest was found, decimal degrees.
|-
|start_date_time||timestamp||NO||When the nest was spotted.
|-
|end_date_time||timestamp||NO||When the nest ceases to be relevant.
|-
|found_by_whom||varchar||YES||Place to credit people involved in finding the nest.
|-
|remarks||text||YES||Free text.
|-
|location||bytearray||YES||Calculated Geometry of the coordinates where the nest was found.
|-
|nest_id||int8||NO||Unique id. Added to allow joins.
|}

=== ''gps.''ee_nest_inhabitant_limited ===
This view shows who lives on which nest, for birds you have access to.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|reference_name||varchar||NO||A string belonging to the nest that you can use to identify that nest. Because it is NOT enforced to be unique, DO NOT USE THIS FIELD FOR JOINING.
|-
|ring_number||varchar||NO||The ring number of a bird on a nest.
|-
|device_info_serial||int8||NO||Following the link from a bird to her track sessions, and considering the nest's start and end time, the bird's trackers within that period are resolved so that they can be shown here. This is the device_info_serial of a tracker.
|-
|key_name||varchar||NO||Following the link from a bird to her track sessions, and considering the nest's start and end time, the track sessions' projects within that period are resolved so that they can be shown here. This is the key_name of a project.
|-
|nest_id||int8||NO||Reference to the nest id. Added to allow joins.
|-
|individual_id||int8||NO||Reference to the individual id. Added to allow joins.
|-
|track_session_id||int8||NO||Reference to a resolved track session id. Added to allow joins.
|}

=== ''gps.''ee_track_session_limited ===

A track session links a tracker and a bird for a given period of time. Therefore, it is '''mandatory for projects''' to supply this piece of '''metadata''' before they can see any tracking data related to a tracker they own.

In order to create a track session, the project must create first an entry in the system for the bird that they are attaching the tracker to, and this is the way for the UvA-BiTS community to make sense of the tracking data that we are receiving. This is why providing the metadata is so important.

By editing the start and end dates, administrators of the project that this track session belongs to, can delimit (and therefore, establish) when the coupling makes sense (or is to be understood as useful).

Whenever a tracker is attached to a bird, a track session must be started (start_date) to indicate that the tracking data that is received is already live data. If the tracker is separated from the bird, the tracking data ceases to make sense, so the track session's end date should be set to denote it has ended. New track sessions can be started if the tracker is re-attached to a bird (possibly the same as before, or maybe a different one) afterwards.

Each row shows track session's information. There exists a row for each track session that your projects allow you to see.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||The project that the track session belongs to.
|-
|device_info_serial||int8||NO||Serial number of the tracker this track session involves.
|-
|ring_number||varchar||NO||Ring number of the bird that this track session involves.
|-
|start_date||timestamp||NO||Date before which the track session is not relevant
|-
|end_date||timestamp||NO||Date at which the track session is not relevant any more
|-
|remarks||text||YES||Free text for comments
|-
|start_latitude||numeric(11,8)||NO||Latitude where the bird was tagged, decimal degrees
|-
|start_longitude||numeric(11,8)||NO||Longitude where the bird was tagged, decimal degrees
|-
|project_id||int8||NO||Reference to the project this track session belongs to. Included just to enable joins.
|-
|tracker_id||int8||NO||Reference to the tracker this track session involves. Included just to enable joins.
|-
|individual_id||int8||NO||Reference to the individual this track session involves. Included just to enable joins.
|-
|track_session_id||int8||NO||Unique identifier of the track session. Included just to enable joins.
|}

== Relation to projects ==

=== ''gps.''ee_user_limited ===

Each row shows a colleague in a project that you belong to, indicating her role and status there.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||Project name
|-
|email||varchar||NO||e-mail address
|-
|login_name||varchar||NO||Login name
|-
|name||varchar||NO||(first) name
|-
|surname||varchar||NO||Surname
|-
|role||varchar||NO||Role in the project
|-
|status||varchar||NO||Status of the membership
|-
|user_id||int8||NO||Unique identifier of the user. Included to enable joins.
|}

=== ''gps.''ee_tracker_ownership_limited ===

Each row that you see expresses that a tracker belongs to one of the projects you are involved with.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||The project that the tracker belongs to.
|-
|device_info_serial||int8||NO||Serial number of the tracker.
|-
|project_id||int8||NO||Reference to the project. Included just to enable joins.
|-
|tracker_id||int8||NO||Reference to the tracker. Included just to enable joins.
|}

=== ''gps.''ee_individual_ownership_limited ===

Each row that you see expresses that a bird belongs to one of the projects you are involved with.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||The project that the individual belongs to.
|-
|ring_number||varchar||NO||Ring number of the bird.
|-
|project_id||int8||NO||Reference to the project. Included just to enable joins.
|-
|individual_id||int8||NO||Reference to the individual. Included just to enable joins.
|}

== Engineering data ==

'''Note:''' These views will show no rows for trackers not yet attached to birds.

=== ''gps.''ee_comm_limited ===

Each row indicates status and result of communications with ground stations for each tracker that your project allows you to see.

Only for newer firmware (v.1.0.1.4 and higher).

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
| device_info_serial || int4 || NO || Serial number of the tracker.
|-
| date_time || timestamp || NO || timestamp of start communication
|-
| date_time_end || timestamp ||YES|| timestamp of end communication
|-
| date_time_utc || timestamp ||YES|| date_time in utc (if available)
|-
| date_time_end_utc || timestamp ||YES|| date_time_end in utc (if available)
|-
| action || text||YES|| JOINED / RESCHEDULE ??
|-
| bytes_communicated || int4 ||YES|| Number of bytes communicated (total of all bytes per DATA line)
|-
|status || text ||YES|| READY, TIMEOUT, LOWPOWER, MISSING
|-
| network_join_time || int4||YES||milliseconds
|-
| number_identical_lines || int4 ||YES|| Number identical lines in data block. Gives indication
|-
| non_ascii_char || int4 ||YES|| Number of non ascii characters (indicated by question-mark ?)
|-
| sts_time || timestamp ||YES|| derived from S0 hh,mi
|-
| timesynced || int2||YES|| TimeSynced (0=not sy:wnced, 1=user set, 2=Gps Synced)
|-
|devicemode || int2||YES||DeviceMode (0-99). This describes whether the device is in a special operational mode. 0 = normal operational mode; 80 hex = Memory save mode (memory is almost full and all activity is drastically reduced so the circular memory probably won't be overwritten or, if it does, it will be kept to a minimum)
|-
| vbat || float8 ||YES|| Battery voltage in V (0-~4.200V)
|-
|logb || int4 ||YES|| logged bytes, waiting for download (0-4Mbyte)
|-
| logp || int2 ||YES|| Logged percentage (units 0.1%) of memory waiting to be downloaded (0-1000)
|-
| gpsii || int4 ||YES|| GpsIval, current (0-65535)
|-
|comii || int4||YES|| ComIval, current (0-65535)
|-
|accii || int4||YES|| AcceleroIval, current (0-65535)
|-
|swddd || int4||YES|| User switches, current (0-65535)
|-
|last_gps_fix ||timestamp||YES||timestamp of last fix
|-
|latitude||float8||YES||Latitude of last fix. Decimal degrees
|-
|longitude||float8||YES||Longitude of last fix. Decimal degrees
|}

=== ''gps.''ee_energy_limited ===

Only for newer firmware.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker
|-
|date_time||timestamp||NO||Date and time of the reported data
|-
|timesynced ||int2||YES||TimeSynced (0=not synced, 1=user set, 2=Gps synced)
|-
|devicemode ||int2||YES||(0-99)
|-
|vsll ||float8||YES||Solar Voltage during charging in mV (0-~2500)
|-
|vbat ||float8||YES||Battery Voltage in V (0-~4.200)
|-
|ssw ||int2||YES|| Solar Cell Switches ... (0-255) fw1022 and up
|-
| ||||YES||extended fields if special user switch is on
|-
|vsoo ||float8||YES|| Solar voltage open in V (0-~2.500)
|-
|vdrf ||float8||YES||reference voltage Dac in V (0-~2.435)
|-
|vchr ||float8||YES||charge current voltage
|-
|g ||int2||YES|| Gps battery threshold OK
|-
|c || int2||YES||Com battery threshold OK
|-
|m || int2||YES||Charge Mode (0,1,2 Com battery threshold OK
|-
|pressure || int4||YES|| in pascal (1mBar = 100Pa, 1pressuremeter = ~12Pa)
|-
|temperature || float8||YES|| in celcius degrees
|}

=== ''gps.''ee_settings_limited ===

Settings are either from C0-C1-C9 lines or parsed from uplink communication lines.
If C0-C1 is parsed its values are compared with the previous value and if the settings are the same, then no new records need to be added.
If uplink settings are parsed, then wait until the last setting has been seen (DL command or EOF); then store all the new settings in one record. Only for newer firmware.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||
|-
|date_time||timestamp||NO||Derived timestamp from Hh:mi fields in C0 combined with computer timestamp in JOINED line
|-
|timesynced || int2 ||YES||
|-
|timezone_time || int2 ||YES||
|-
|swddd || int4||YES|| SpecialUserSwitches
|-
|gpsnavmode || int2||YES|| GpsNavMode 0-8. This is the Kalman filter used for GPS navigation mode often called the "Dynamic platform model". The following codes are provided by the GPS chip manufacturer for the UBlox GPS chip LEAS4S: 1 = stationary, 2 = pedestrian, 3 = automotive, 4 = sea, 5 = airborne 1G, 6 = airborne 2G, 7 = airborne 3G. The UBlox GPS chip LEAS6S uses the following codes: 0 = portable, 2 = stationary, 3 = pedestrian, 4 = automotive, 5 = sea, 6 = airborne with < 1g acceleration, 7 = airborne with < 2g acceleration, 8 = airborne with < 4g acceleration
|-
|gpsreceivermode || int2 ||YES|| GpsReceiverNode 0-3
|-
|gpsfixtime || int2||YES|| GpsFixTime 0-255
|-
|gps_ivaln_on_h || int2||YES|| hours Gps IvalN on
|-
|gps_ivaln_on_m || int2||YES|| minutes Gps IvalN on
|-
|gps_ivaln || int4||YES|| GpsIvalN (0-65530)
|-
|gps_ivale_on_h || int2||YES|| hours Gps IvalE on
|-
|gps_ivale_on_m || int2||YES|| minutes GpS IvalE on
|-
|gps_ivale || int4||YES|| GpsIvalE
|-
|com_ivaln_on_h || int2||YES|| hours Com IvalN on
|-
|com_ivaln_on_m || int2||YES|| minutes Com IvalN on
|-
|com_ivaln || int4||YES|| ComIvalN (0-65530)
|-
|com_ivale_on_h || int2||YES|| hours Com IvalE on
|-
|com_ivale_on_m || int2||YES|| minutes Com IvalE on
|-
|com_ivale || int4||YES|| ComIvalE
|-
|accii || int4||YES|| Accelero measurement interval
|-
|accbn || int4||YES|| Accelero number of blocks
|-
|accsn || int4||YES|| Accelero number of samples (0-65536)
|-
|f || int2||YES||Accelero sample frequency (0=20Hz, 1=10Hz, 2=5Hz, 3=1Hz)
|-
|pdp || int2||YES|| PdopMsk (3-250)
|-
|pac||int2||YES|| PaccMsk (10-100)
|-
|tdp ||int2||YES|| TdopMsk (3-250)
|-
|tac ||int2||YES|| TaccMsk (30-300)
|-
|ga || int2||YES|| Additional Fixes GpsIval mode (3-25) before interval fix is logged
|-
|aol || int2||YES||AccMemOffLev (100 - 950) units 0.1%; if Memfull above this level all Accelero activity will be overruled and switched off untill MemFull < level - 5
|-
|eventnum || text||YES|| 12 eventnumbers (0-0xff) only printed if not zero, therfore converted to text
|}

=== ''gps.''ee_trackingfile_limited ===

Each row shows the status of the last attempt to parse (process) the raw data of a file for a given tracker, for each tracker that your projects allow you to see.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|id||int8||NO||Unique identifier of the row
|-
|date_reported||date||NO||Date that the file arrived
|-
|filename||varchar||NO||Name of the file
|-
|last_modified_date||timestamp||NO||Date that the file was last modified on
|-
|device_info_serial||int4||YES||Serial number of the tracker this file comes from
|-
|filesize||int8||NO||Size of the file in KB
|-
|parsed_date||timestamp||YES||Date and time that the file was last attempted to be parsed on
|-
|version||int4||NO||Optimistic locking field. Ignore it as a user.
|-
|last_found_date||timestamp||YES||Last time that a process that runs to look for new files saw the file
|-
|n_lines||int4||YES||Count of lines found in the file last time it was processed
|-
|n_data_lines||int4||YES||Count of actual data lines, out of the total lines
|-
|n_tracking_records||int4||YES||Count of GPS fixes found during the parsing process
|-
|n_communication_records||int4||YES||Count of communication records found during the parsing process
|-
|n_energy_records||int4||YES||Count of energy records found during the parsing process
|-
|n_acceleration_records||int4||YES||Count of acceleration records found during the parsing process
|-
|n_acceleration_start_records||int4||YES||Count of acceleration start records found during the parsing process
|-
|n_errors||int4||YES||Count of errors encountered during the parsing process
|-
|n_settings_records||int4||YES||Count of settings records found during the parsing process
|-
|filepath||text||NO||Directory path of the file on the server
|}

=== ''gps.''ee_warning_limited ===

Since firmware version 1.0.2.7 this table was added for events that do not occur very often (C10, C20) or errors that occur during parsing.

For example all CRC failures can be selected with :

select * from gps.ee_warning_limited
where warning = 'CRC8 failure'
order by device_info_serial;

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker
|-
|filename||text||NO||File name
|-
|linenumber||integer||YES||line number in file
|-
|warning||text||YES||The warning message
|-
|line||text||YES||The actual line contents in the file
|}

== Sharings ==

Projects can now share trackers with other projects during periods of time. This means that people in the receiving projects can access information that these shared trackers provide during those time periods.

The following views allow you to see this shared information:

* Tracking data
** ''gps.''ee_shared_tracking_speed_limited: tracking information (gps, speed), during the shared period, for trackers shared
** ''gps.''ee_shared_acc_start_limited: acceleration start information, during the shared period, for trackers shared
** ''gps.''ee_shared_acceleration_limited: acceleration information for trackers shared
* ''Meta'' data
** ''gps.''ee_shared_tracker_limited: information about each shared tracker
** ''gps.''ee_shared_individual_limited: information about birds with shared trackers
** ''gps.''ee_shared_track_session_limited: information about track sessions which involve shared trackers
* Relation to projects
** ''gps.''ee_shared_tracker_sharing_limited: view information for each sharing that your projects allow you to see

== Accelerometer calibration ==

The accelerometer measures the acceleration in 3D (20 times per second). Acceleration is caused by gravity (static component) and by a change in speed (dynamic component).
When the logger is static in horizontal position (solar cells up), gravity (green arrow) shows a heave (vertical acceleration, z) of +1 g (1 g = 9.81 m.s-2) while surge and heave are both 0. When the front of the logger is vertically pointing down (forward acceleration, x), surge is +1g and when it is rolled to the right (sidewards acceleration, y) sway is +1g. Due to gravity, the pitch of a logger is thus translated into the values of heave, surge and sway (or z, x, y).

However, the values are measured in mV instead of g. Before delivering the GPS-loggers, they have been calibrated at UvA. The acceleration (A) expressed in g is calculated from the measured value (M) and the calibration factors offset (''O'') and sensitivity (''S'').

Thus acceleration for heave (z), surge (x) and sway (y) is calculated as:
Az = (Mz-Oz)/Sz ; Ax = (Mx-Ox)/Sx; Ay = (My-Oy)/Sy ;

The values of ''O'' (in general ranging from 0-60, sometimes even 300) and ''S'' (ranging from 1300-1450) are available in the device table of the data base. If not available, use the factory defaults ''O'' = 0 and ''S'' = 1365.

{|
| [[File:accelero1.png|300px]]||[[File:accelero2.png|300px]]|
|}

In the figure above heave (z) is shown in green, surge(x) in red and sway (y) in blue, all values are expressed in g (on the y-axis).

Useful Queries

2014-12-09T13:48:06Z

Judy: /* get_uvagps_track_speed */

Useful Queries

2014-12-09T13:46:07Z

Judy: /* Calculating summary statistics */

UvA-BiTS Tracking Data

2014-12-09T13:42:54Z

Judy: /* gps.ee_track_session_limited */

== ''Meta'' data ==

=== ''gps.''ee_project_limited ===

Each row shows the details of each project, for the projects you are allowed to see.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||Unique key name, with the form <species>_<place>
|-
|station_name||varchar||NO||Where the receiving station is
|-
|start_date||timestamp||NO||Project start date
|-
|end_date||timestamp||NO||Date at which the project is not relevant any more
|-
|description||text||YES||String for free description. Updatable by the user.
|-
|project_id||int8||NO||Project unique id. Included just to enable joins.
|-
|parent_id||int8||YES||Reference to the parent project in the hierarchy.
|}

=== ''gps.''ee_tracker_limited ===

Each row shows tracker details for all trackers that your projects allow you to see.

This view replaces the old gps.uva_device_limited.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int8||NO||Serial number of the tracker
|-
|firmware_version||varchar||YES||String for firmware version
|-
|mass||numeric(4,2)||YES||Mass (in grams)
|-
|start_date||timestamp||NO||Date before which the device is not relevant
|-
|end_date||timestamp||NO||Date at which the device is not relevant any more
|-
|x_o||numeric(30,6)||YES||Accelerometer calibration: offset on the x (or surge) axis
|-
|x_s||numeric(30,6)||YES||Accelerometer calibration: sensitivity on the x (or surge) axis
|-
|y_o||numeric(30,6)||YES||Accelerometer calibration: offset on the y (or sway) axis
|-
|y_s||numeric(30,6)||YES||Accelerometer calibration: sensitivity on the y (or sway) axis
|-
|z_o||numeric(30,6)||YES||Accelerometer calibration: offset on the z (or heave) axis
|-
|z_s||numeric(30,6)||YES||Accelerometer calibration: sensitivity on the y (or heave) axis
|-
|tracker_id||int8||NO||Tracker unique id. Included just to enable joins.
|}

=== ''gps.''ee_individual_limited ===

Each row shows bird's details for all birds that your projects allow you to see.

This view replaces the old gps.uva_individual_limited.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|ring_number||varchar||NO||Ring number.
|-
|species_latin_name||varchar||NO||Latin name of the species of the bird.
|-
|colour_ring||varchar||YES||String for ring colour.
|-
|sex||varchar||NO||Sex (M=male, F=female, X=unknown/unspecified)
|-
|mass||numeric(5,0)||YES||Mass (grams)
|-
|start_date||timestamp||NO||Date when the bird was first tracked with an UvA-BiTS logger.
|-
|end_date||timestamp||NO||Date at which the bird is no longer relevant for the tracking project.
|-
|remarks||text||YES||Text for user remarks
|-
|individual_id||int8||NO||Bird unique id. Included just to enable joins.
|}

=== ''gps.''ee_nest_limited ===
This view shows nests you have access to. A nest is always for a bird with a tracker (so a track session must exist for that bird), and must contain a short name (a reference name for you to use) and a latitude and longitude pair.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|reference_name||varchar||NO||A string with a useful reference for you. We recommend its being unique for each nest in a project, but the system does NOT enforce it.
|-
|latitude||numeric||YES||Latitude of the coordinates where the nest was found, decimal degrees.
|-
|longitude||numeric||YES||Longitude of the coordinates where the nest was found, decimal degrees.
|-
|start_date_time||timestamp||NO||When the nest was spotted.
|-
|end_date_time||timestamp||NO||When the nest ceases to be relevant.
|-
|found_by_whom||varchar||YES||Place to credit people involved in finding the nest.
|-
|remarks||text||YES||Free text.
|-
|location||bytearray||YES||Calculated Geometry of the coordinates where the nest was found.
|-
|nest_id||int8||NO||Unique id. Added to allow joins.
|}

=== ''gps.''ee_nest_inhabitant_limited ===
This view shows who lives on which nest, for birds you have access to.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|reference_name||varchar||NO||A string belonging to the nest that you can use to identify that nest. Because it is NOT enforced to be unique, DO NOT USE THIS FIELD FOR JOINING.
|-
|ring_number||varchar||NO||The ring number of a bird on a nest.
|-
|device_info_serial||int8||NO||Following the link from a bird to her track sessions, and considering the nest's start and end time, the bird's trackers within that period are resolved so that they can be shown here. This is the device_info_serial of a tracker.
|-
|key_name||varchar||NO||Following the link from a bird to her track sessions, and considering the nest's start and end time, the track sessions' projects within that period are resolved so that they can be shown here. This is the key_name of a project.
|-
|nest_id||int8||NO||Reference to the nest id. Added to allow joins.
|-
|individual_id||int8||NO||Reference to the individual id. Added to allow joins.
|-
|track_session_id||int8||NO||Reference to a resolved track session id. Added to allow joins.
|}

=== ''gps.''ee_track_session_limited ===

A track session links a tracker and a bird for a given period of time. Therefore, it is '''mandatory for projects''' to supply this piece of '''metadata''' before they can see any tracking data related to a tracker they own.

In order to create a track session, the project must create first an entry in the system for the bird that they are attaching the tracker to, and this is the way for the UvA-BiTS community to make sense of the tracking data that we are receiving. This is why providing the metadata is so important.

By editing the start and end dates, administrators of the project that this track session belongs to, can delimit (and therefore, establish) when the coupling makes sense (or is to be understood as useful).

Whenever a tracker is attached to a bird, a track session must be started (start_date) to indicate that the tracking data that is received is already live data. If the tracker is separated from the bird, the tracking data ceases to make sense, so the track session's end date should be set to denote it has ended. New track sessions can be started if the tracker is re-attached to a bird (possibly the same as before, or maybe a different one) afterwards.

Each row shows track session's information. There exists a row for each track session that your projects allow you to see.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||The project that the track session belongs to.
|-
|device_info_serial||int8||NO||Serial number of the tracker this track session involves.
|-
|ring_number||varchar||NO||Ring number of the bird that this track session involves.
|-
|start_date||timestamp||NO||Date before which the track session is not relevant
|-
|end_date||timestamp||NO||Date at which the track session is not relevant any more
|-
|remarks||text||YES||Free text for comments
|-
|start_latitude||numeric(11,8)||NO||Latitude where the bird was tagged, decimal degrees
|-
|start_longitude||numeric(11,8)||NO||Longitude where the bird was tagged, decimal degrees
|-
|project_id||int8||NO||Reference to the project this track session belongs to. Included just to enable joins.
|-
|tracker_id||int8||NO||Reference to the tracker this track session involves. Included just to enable joins.
|-
|individual_id||int8||NO||Reference to the individual this track session involves. Included just to enable joins.
|-
|track_session_id||int8||NO||Unique identifier of the track session. Included just to enable joins.
|}

== Tracking data ==

'''Note:''' These views will show no rows for trackers not yet attached to birds.

=== ''gps.''ee_tracking_speed_limited ===

Shows the tracking data that your projects allow you to see.

Each row represents a GPS fix as it is interpreted from the data that comes via the GPS tracker, plus it adds speeds.

It replaces old views gps.uva_tracking_limited, gps.uva_tracking_speed_3d_limited and gps.uva_tracking_speed. This table structure has ''not'' changed with respect to our previous database.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker that reported the fix. Part of the unique identifier of the GPS fix.
|-
|date_time||timestamp||NO||Time when the fix was taken. UTC (i.e.: no timezone). Part of the unique identifier of the GPS fix.
|-
|latitude||float8||YES||Latitude of the GPS fix, decimal degrees.
|-
|longitude||float8||YES||Longitude of the GPS fix, decimal degrees.
|-
|altitude||int4||YES||Altitude above sea level measured by GPS tag in meters
|-
|pressure||int4||YES||Pressure measured by GPS tag sensor in Pascals
|-
|temperature||float8||YES||Temperature measured by GPS tag sensor in Celsius
|-
|h_accuracy||float8||YES||Horizontal accuracy (meters)
|-
|v_accuracy||float8||YES||vertical accuracy (meters)
|-
|x_speed||float8||YES||meters/second
|-
|y_speed||float8||YES||meters/second
|-
|z_speed||float8||YES||meters/second
|-
|gps_fixtime||float8||YES||GPS fix time (also known as Time to Fix) in seconds
|-
|userflag||int4||YES||Acceptable = 0; Data flagged as unacceptable by user if not equal to 0.
|-
|satellites_used||int2||YES||Number of satellites used for fix the GPS measurement
|-
|positiondop||float8||YES||Positional DOP (Dilution of Precision), unitless
|-
|speed_accuracy||float8||YES||
|-
|location||geometry||YES||postgreSQL geometry; to allow GIS operations
|-
|vnorth||float8||YES||- vx * sin(lat) * cos(long) - vy * sin(lat) * sin (long) + vz * cos(lat), m/s
|-
|veast||float8||YES||- vx * sin(lon) + vy * cos(lon), m/s
|-
|vdown||float8||YES||- vx * cos(lat) * cos(lon) - vy * cos(lat) * sin(lon) - vz * sin(lat), m/s
|-
|speed||float8||YES||- vx * sin(lon) + vy * cos(lon)
|-
|speed_3d||float8||YES|| sqrt(d.x_speed^2 + d.y_speed^2 + d.z_speed^2), m/s
|-
|speed_2d||float8||YES|| Speed on 2D, m/s
|-
|direction||numeric||YES|| Heading in degrees; 0 is north, 90 is east
|-
|altitude_agl||float8||YES|| Reported GPS altitude MINUS terrain elevation on the coordinate based on Digital Elevation Model data used in the e-ecology database (for further information see: [[https://services.e-ecology.sara.nl/wiki/index.php/Elevation]]
|}

=== ''gps.''ee_acceleration_limited ===

Each row represents an acceleration measurement as it is interpreted from the data that comes via the accelerometer. Acceleration is looked at in batches for a period of time, for 3 components (X,Y,Z). The blocks are delimited by a starting date and time, and then each observation is numbered consecutively.

The logger tries to mark the start of blocks from the GPS fix that coincides with the beginning of the acceleration block. If there's no GPS fix during that block, then it extrapolates the date and time and makes a special record which is interpreted by our accelerometer processing software and you can find it via gps.ee_acc_start_limited. Therefore, to know all valid dates and times that mark a valid block for a tracker, you need to consider all of the values in column date_time on gps.ee_tracking_speed_limited for the given tracker, column date_time on gps.ee_acc_start_limited, and all the values on column date_time for that tracker. And the order is important within each block, so you should sort the acceleration measurements by block and index.

This table structure has ''not'' changed with respect to our previous database.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker that reported the fix. Part of the unique identifier of the GPS fix.
|-
|date_time||timestamp||NO||Time of the acceleration block that this measurement belongs to.
|-
|index||int2||NO||Sorting order of the measurement within the acceleration block (each block starts at 0).
|-
|x_acceleration||int2||YES||Acceleration measured on the x (or surge) axis (range between -2666 and 2666).
|-
|y_acceleration||int2||YES||Acceleration measured on the y (or sway) axis (range between -2666 and 2666).
|-
|z_acceleration||int2||YES||Acceleration measured on the z (or heave) axis (range between -2666 and 2666).
|}
=== ''gps.''ee_acc_start_limited ===

Each row represents the start of a block of acceleration measurements when a GPS fix is not available at that acceleration block's time.

The logger tries to mark the start of blocks from the GPS fix that coincides with the begin of the acceleration block. If there's no GPS fix during that block, then it extrapolates the date and time and makes a special record which is interpreted by our accelerometer processing software and made available through this view.

Therefore, to know all valid dates and times that mark a valid block for a tracker, you need to consider all of the values in column date_time on gps.ee_tracking_speed_limited for the given tracker, column date_time on gps.ee_acc_start_limited, and all the values on column date_time for that tracker. And the order is important within each block, so you should sort the acceleration measurements by block and index.

Added in firmware 1.0.2.7.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker that reported the acceleration measurement.
|-
|date_time||timestamp||NO||Time to mark the start of the acceleration block.
|-
|line_counter||int4||NO||Number of lines associated with this set of accelerations.
|-
|timesynced||int2||YES||The source of the time that marks the acceleration block (0=not synced, 1=user set, 2=Gps Synced).
|-
|ascii||int4||YES||Acceleration measurement interval.
|-
|accsn||int4||YES||Number of samples (0-65536).
|-
|f||int2||YES||Acceleration sampling frequency (0=20Hz, 1=10Hz, 2=5Hz, 3=1Hz).
|}

== Relation to projects ==

=== ''gps.''ee_user_limited ===

Each row shows a colleague in a project that you belong to, indicating her role and status there.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||Project name
|-
|email||varchar||NO||e-mail address
|-
|login_name||varchar||NO||Login name
|-
|name||varchar||NO||(first) name
|-
|surname||varchar||NO||Surname
|-
|role||varchar||NO||Role in the project
|-
|status||varchar||NO||Status of the membership
|-
|user_id||int8||NO||Unique identifier of the user. Included to enable joins.
|}

=== ''gps.''ee_tracker_ownership_limited ===

Each row that you see expresses that a tracker belongs to one of the projects you are involved with.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||The project that the tracker belongs to.
|-
|device_info_serial||int8||NO||Serial number of the tracker.
|-
|project_id||int8||NO||Reference to the project. Included just to enable joins.
|-
|tracker_id||int8||NO||Reference to the tracker. Included just to enable joins.
|}

=== ''gps.''ee_individual_ownership_limited ===

Each row that you see expresses that a bird belongs to one of the projects you are involved with.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||The project that the individual belongs to.
|-
|ring_number||varchar||NO||Ring number of the bird.
|-
|project_id||int8||NO||Reference to the project. Included just to enable joins.
|-
|individual_id||int8||NO||Reference to the individual. Included just to enable joins.
|}

== Engineering data ==

'''Note:''' These views will show no rows for trackers not yet attached to birds.

=== ''gps.''ee_comm_limited ===

Each row indicates status and result of communications with ground stations for each tracker that your project allows you to see.

Only for newer firmware (v.1.0.1.4 and higher).

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
| device_info_serial || int4 || NO || Serial number of the tracker.
|-
| date_time || timestamp || NO || timestamp of start communication
|-
| date_time_end || timestamp ||YES|| timestamp of end communication
|-
| date_time_utc || timestamp ||YES|| date_time in utc (if available)
|-
| date_time_end_utc || timestamp ||YES|| date_time_end in utc (if available)
|-
| action || text||YES|| JOINED / RESCHEDULE ??
|-
| bytes_communicated || int4 ||YES|| Number of bytes communicated (total of all bytes per DATA line)
|-
|status || text ||YES|| READY, TIMEOUT, LOWPOWER, MISSING
|-
| network_join_time || int4||YES||milliseconds
|-
| number_identical_lines || int4 ||YES|| Number identical lines in data block. Gives indication
|-
| non_ascii_char || int4 ||YES|| Number of non ascii characters (indicated by question-mark ?)
|-
| sts_time || timestamp ||YES|| derived from S0 hh,mi
|-
| timesynced || int2||YES|| TimeSynced (0=not sy:wnced, 1=user set, 2=Gps Synced)
|-
|devicemode || int2||YES||DeviceMode (0-99). This describes whether the device is in a special operational mode. 0 = normal operational mode; 80 hex = Memory save mode (memory is almost full and all activity is drastically reduced so the circular memory probably won't be overwritten or, if it does, it will be kept to a minimum)
|-
| vbat || float8 ||YES|| Battery voltage in V (0-~4.200V)
|-
|logb || int4 ||YES|| logged bytes, waiting for download (0-4Mbyte)
|-
| logp || int2 ||YES|| Logged percentage (units 0.1%) of memory waiting to be downloaded (0-1000)
|-
| gpsii || int4 ||YES|| GpsIval, current (0-65535)
|-
|comii || int4||YES|| ComIval, current (0-65535)
|-
|accii || int4||YES|| AcceleroIval, current (0-65535)
|-
|swddd || int4||YES|| User switches, current (0-65535)
|-
|last_gps_fix ||timestamp||YES||timestamp of last fix
|-
|latitude||float8||YES||Latitude of last fix. Decimal degrees
|-
|longitude||float8||YES||Longitude of last fix. Decimal degrees
|}

=== ''gps.''ee_energy_limited ===

Only for newer firmware.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker
|-
|date_time||timestamp||NO||Date and time of the reported data
|-
|timesynced ||int2||YES||TimeSynced (0=not synced, 1=user set, 2=Gps synced)
|-
|devicemode ||int2||YES||(0-99)
|-
|vsll ||float8||YES||Solar Voltage during charging in mV (0-~2500)
|-
|vbat ||float8||YES||Battery Voltage in V (0-~4.200)
|-
|ssw ||int2||YES|| Solar Cell Switches ... (0-255) fw1022 and up
|-
| ||||YES||extended fields if special user switch is on
|-
|vsoo ||float8||YES|| Solar voltage open in V (0-~2.500)
|-
|vdrf ||float8||YES||reference voltage Dac in V (0-~2.435)
|-
|vchr ||float8||YES||charge current voltage
|-
|g ||int2||YES|| Gps battery threshold OK
|-
|c || int2||YES||Com battery threshold OK
|-
|m || int2||YES||Charge Mode (0,1,2 Com battery threshold OK
|-
|pressure || int4||YES|| in pascal (1mBar = 100Pa, 1pressuremeter = ~12Pa)
|-
|temperature || float8||YES|| in celcius degrees
|}

=== ''gps.''ee_settings_limited ===

Settings are either from C0-C1-C9 lines or parsed from uplink communication lines.
If C0-C1 is parsed its values are compared with the previous value and if the settings are the same, then no new records need to be added.
If uplink settings are parsed, then wait until the last setting has been seen (DL command or EOF); then store all the new settings in one record. Only for newer firmware.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||
|-
|date_time||timestamp||NO||Derived timestamp from Hh:mi fields in C0 combined with computer timestamp in JOINED line
|-
|timesynced || int2 ||YES||
|-
|timezone_time || int2 ||YES||
|-
|swddd || int4||YES|| SpecialUserSwitches
|-
|gpsnavmode || int2||YES|| GpsNavMode 0-8. This is the Kalman filter used for GPS navigation mode often called the "Dynamic platform model". The following codes are provided by the GPS chip manufacturer for the UBlox GPS chip LEAS4S: 1 = stationary, 2 = pedestrian, 3 = automotive, 4 = sea, 5 = airborne 1G, 6 = airborne 2G, 7 = airborne 3G. The UBlox GPS chip LEAS6S uses the following codes: 0 = portable, 2 = stationary, 3 = pedestrian, 4 = automotive, 5 = sea, 6 = airborne with < 1g acceleration, 7 = airborne with < 2g acceleration, 8 = airborne with < 4g acceleration
|-
|gpsreceivermode || int2 ||YES|| GpsReceiverNode 0-3
|-
|gpsfixtime || int2||YES|| GpsFixTime 0-255
|-
|gps_ivaln_on_h || int2||YES|| hours Gps IvalN on
|-
|gps_ivaln_on_m || int2||YES|| minutes Gps IvalN on
|-
|gps_ivaln || int4||YES|| GpsIvalN (0-65530)
|-
|gps_ivale_on_h || int2||YES|| hours Gps IvalE on
|-
|gps_ivale_on_m || int2||YES|| minutes GpS IvalE on
|-
|gps_ivale || int4||YES|| GpsIvalE
|-
|com_ivaln_on_h || int2||YES|| hours Com IvalN on
|-
|com_ivaln_on_m || int2||YES|| minutes Com IvalN on
|-
|com_ivaln || int4||YES|| ComIvalN (0-65530)
|-
|com_ivale_on_h || int2||YES|| hours Com IvalE on
|-
|com_ivale_on_m || int2||YES|| minutes Com IvalE on
|-
|com_ivale || int4||YES|| ComIvalE
|-
|accii || int4||YES|| Accelero measurement interval
|-
|accbn || int4||YES|| Accelero number of blocks
|-
|accsn || int4||YES|| Accelero number of samples (0-65536)
|-
|f || int2||YES||Accelero sample frequency (0=20Hz, 1=10Hz, 2=5Hz, 3=1Hz)
|-
|pdp || int2||YES|| PdopMsk (3-250)
|-
|pac||int2||YES|| PaccMsk (10-100)
|-
|tdp ||int2||YES|| TdopMsk (3-250)
|-
|tac ||int2||YES|| TaccMsk (30-300)
|-
|ga || int2||YES|| Additional Fixes GpsIval mode (3-25) before interval fix is logged
|-
|aol || int2||YES||AccMemOffLev (100 - 950) units 0.1%; if Memfull above this level all Accelero activity will be overruled and switched off untill MemFull < level - 5
|-
|eventnum || text||YES|| 12 eventnumbers (0-0xff) only printed if not zero, therfore converted to text
|}

=== ''gps.''ee_trackingfile_limited ===

Each row shows the status of the last attempt to parse (process) the raw data of a file for a given tracker, for each tracker that your projects allow you to see.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|id||int8||NO||Unique identifier of the row
|-
|date_reported||date||NO||Date that the file arrived
|-
|filename||varchar||NO||Name of the file
|-
|last_modified_date||timestamp||NO||Date that the file was last modified on
|-
|device_info_serial||int4||YES||Serial number of the tracker this file comes from
|-
|filesize||int8||NO||Size of the file in KB
|-
|parsed_date||timestamp||YES||Date and time that the file was last attempted to be parsed on
|-
|version||int4||NO||Optimistic locking field. Ignore it as a user.
|-
|last_found_date||timestamp||YES||Last time that a process that runs to look for new files saw the file
|-
|n_lines||int4||YES||Count of lines found in the file last time it was processed
|-
|n_data_lines||int4||YES||Count of actual data lines, out of the total lines
|-
|n_tracking_records||int4||YES||Count of GPS fixes found during the parsing process
|-
|n_communication_records||int4||YES||Count of communication records found during the parsing process
|-
|n_energy_records||int4||YES||Count of energy records found during the parsing process
|-
|n_acceleration_records||int4||YES||Count of acceleration records found during the parsing process
|-
|n_acceleration_start_records||int4||YES||Count of acceleration start records found during the parsing process
|-
|n_errors||int4||YES||Count of errors encountered during the parsing process
|-
|n_settings_records||int4||YES||Count of settings records found during the parsing process
|-
|filepath||text||NO||Directory path of the file on the server
|}

=== ''gps.''ee_warning_limited ===

Since firmware version 1.0.2.7 this table was added for events that do not occur very often (C10, C20) or errors that occur during parsing.

For example all CRC failures can be selected with :

select * from gps.ee_warning_limited
where warning = 'CRC8 failure'
order by device_info_serial;

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker
|-
|filename||text||NO||File name
|-
|linenumber||integer||YES||line number in file
|-
|warning||text||YES||The warning message
|-
|line||text||YES||The actual line contents in the file
|}

== Sharings ==

Projects can now share trackers with other projects during periods of time. This means that people in the receiving projects can access information that these shared trackers provide during those time periods.

The following views allow you to see this shared information:

* ''Leaf'' data
** ''gps.''ee_shared_tracker_limited: information about each shared tracker
** ''gps.''ee_shared_individual_limited: information about birds with shared trackers
** ''gps.''ee_shared_track_session_limited: information about track sessions which involve shared trackers
* Tracking data
** ''gps.''ee_shared_tracking_speed_limited: tracking information (gps, speed), during the shared period, for trackers shared
** ''gps.''ee_shared_acc_start_limited: acceleration start information, during the shared period, for trackers shared
** ''gps.''ee_shared_acceleration_limited: acceleration information for trackers shared
* Relation to projects
** ''gps.''ee_shared_tracker_sharing_limited: view information for each sharing that your projects allow you to see

UvA-BiTS Tracking Data

2014-12-09T13:42:06Z

Judy: /* gps.ee_nest_limited */

== ''Meta'' data ==

=== ''gps.''ee_project_limited ===

Each row shows the details of each project, for the projects you are allowed to see.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||Unique key name, with the form <species>_<place>
|-
|station_name||varchar||NO||Where the receiving station is
|-
|start_date||timestamp||NO||Project start date
|-
|end_date||timestamp||NO||Date at which the project is not relevant any more
|-
|description||text||YES||String for free description. Updatable by the user.
|-
|project_id||int8||NO||Project unique id. Included just to enable joins.
|-
|parent_id||int8||YES||Reference to the parent project in the hierarchy.
|}

=== ''gps.''ee_tracker_limited ===

Each row shows tracker details for all trackers that your projects allow you to see.

This view replaces the old gps.uva_device_limited.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int8||NO||Serial number of the tracker
|-
|firmware_version||varchar||YES||String for firmware version
|-
|mass||numeric(4,2)||YES||Mass (in grams)
|-
|start_date||timestamp||NO||Date before which the device is not relevant
|-
|end_date||timestamp||NO||Date at which the device is not relevant any more
|-
|x_o||numeric(30,6)||YES||Accelerometer calibration: offset on the x (or surge) axis
|-
|x_s||numeric(30,6)||YES||Accelerometer calibration: sensitivity on the x (or surge) axis
|-
|y_o||numeric(30,6)||YES||Accelerometer calibration: offset on the y (or sway) axis
|-
|y_s||numeric(30,6)||YES||Accelerometer calibration: sensitivity on the y (or sway) axis
|-
|z_o||numeric(30,6)||YES||Accelerometer calibration: offset on the z (or heave) axis
|-
|z_s||numeric(30,6)||YES||Accelerometer calibration: sensitivity on the y (or heave) axis
|-
|tracker_id||int8||NO||Tracker unique id. Included just to enable joins.
|}

=== ''gps.''ee_individual_limited ===

Each row shows bird's details for all birds that your projects allow you to see.

This view replaces the old gps.uva_individual_limited.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|ring_number||varchar||NO||Ring number.
|-
|species_latin_name||varchar||NO||Latin name of the species of the bird.
|-
|colour_ring||varchar||YES||String for ring colour.
|-
|sex||varchar||NO||Sex (M=male, F=female, X=unknown/unspecified)
|-
|mass||numeric(5,0)||YES||Mass (grams)
|-
|start_date||timestamp||NO||Date when the bird was first tracked with an UvA-BiTS logger.
|-
|end_date||timestamp||NO||Date at which the bird is no longer relevant for the tracking project.
|-
|remarks||text||YES||Text for user remarks
|-
|individual_id||int8||NO||Bird unique id. Included just to enable joins.
|}

=== ''gps.''ee_nest_limited ===
This view shows nests you have access to. A nest is always for a bird with a tracker (so a track session must exist for that bird), and must contain a short name (a reference name for you to use) and a latitude and longitude pair.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|reference_name||varchar||NO||A string with a useful reference for you. We recommend its being unique for each nest in a project, but the system does NOT enforce it.
|-
|latitude||numeric||YES||Latitude of the coordinates where the nest was found, decimal degrees.
|-
|longitude||numeric||YES||Longitude of the coordinates where the nest was found, decimal degrees.
|-
|start_date_time||timestamp||NO||When the nest was spotted.
|-
|end_date_time||timestamp||NO||When the nest ceases to be relevant.
|-
|found_by_whom||varchar||YES||Place to credit people involved in finding the nest.
|-
|remarks||text||YES||Free text.
|-
|location||bytearray||YES||Calculated Geometry of the coordinates where the nest was found.
|-
|nest_id||int8||NO||Unique id. Added to allow joins.
|}

=== ''gps.''ee_nest_inhabitant_limited ===
This view shows who lives on which nest, for birds you have access to.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|reference_name||varchar||NO||A string belonging to the nest that you can use to identify that nest. Because it is NOT enforced to be unique, DO NOT USE THIS FIELD FOR JOINING.
|-
|ring_number||varchar||NO||The ring number of a bird on a nest.
|-
|device_info_serial||int8||NO||Following the link from a bird to her track sessions, and considering the nest's start and end time, the bird's trackers within that period are resolved so that they can be shown here. This is the device_info_serial of a tracker.
|-
|key_name||varchar||NO||Following the link from a bird to her track sessions, and considering the nest's start and end time, the track sessions' projects within that period are resolved so that they can be shown here. This is the key_name of a project.
|-
|nest_id||int8||NO||Reference to the nest id. Added to allow joins.
|-
|individual_id||int8||NO||Reference to the individual id. Added to allow joins.
|-
|track_session_id||int8||NO||Reference to a resolved track session id. Added to allow joins.
|}

=== ''gps.''ee_track_session_limited ===

A track session links a tracker and a bird for a given period of time. Therefore, it is '''mandatory for projects''' to supply this piece of '''metadata''' before they can see any tracking data related to a tracker they own.

In order to create a track session, the project must create first an entry in the system for the bird that they are attaching the tracker to, and this is the way for the UvA-BiTS community to make sense of the tracking data that we are receiving. This is why providing the metadata is so important.

By editing the start and end dates, administrators of the project that this track session belongs to, can delimit (and therefore, establish) when the coupling makes sense (or is to be understood as useful).

Whenever a tracker is attached to a bird, a track session must be started (start_date) to indicate that the tracking data that is received is already live data. If the tracker is separated from the bird, the tracking data ceases to make sense, so the track session's end date should be set to denote it has ended. New track sessions can be started if the tracker is re-attached to a bird (possibly the same as before, or maybe a different one) afterwards.

Each row shows track session's information. There exists a row for each track session that your projects allow you to see.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||The project that the track session belongs to.
|-
|device_info_serial||int8||NO||Serial number of the tracker this track session involves.
|-
|ring_number||varchar||NO||Ring number of the bird that this track session involves.
|-
|start_date||timestamp||NO||Date before which the track session is not relevant
|-
|end_date||timestamp||NO||Date at which the track session is not relevant any more
|-
|remarks||text||YES||Free text for comments
|-
|start_latitude||numeric(11,8)||NO||Latitude where the bird was tagged
|-
|start_longitude||numeric(11,8)||NO||Longitude where the bird was tagged
|-
|project_id||int8||NO||Reference to the project this track session belongs to. Included just to enable joins.
|-
|tracker_id||int8||NO||Reference to the tracker this track session involves. Included just to enable joins.
|-
|individual_id||int8||NO||Reference to the individual this track session involves. Included just to enable joins.
|-
|track_session_id||int8||NO||Unique identifier of the track session. Included just to enable joins.
|}

== Tracking data ==

'''Note:''' These views will show no rows for trackers not yet attached to birds.

=== ''gps.''ee_tracking_speed_limited ===

Shows the tracking data that your projects allow you to see.

Each row represents a GPS fix as it is interpreted from the data that comes via the GPS tracker, plus it adds speeds.

It replaces old views gps.uva_tracking_limited, gps.uva_tracking_speed_3d_limited and gps.uva_tracking_speed. This table structure has ''not'' changed with respect to our previous database.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker that reported the fix. Part of the unique identifier of the GPS fix.
|-
|date_time||timestamp||NO||Time when the fix was taken. UTC (i.e.: no timezone). Part of the unique identifier of the GPS fix.
|-
|latitude||float8||YES||Latitude of the GPS fix, decimal degrees.
|-
|longitude||float8||YES||Longitude of the GPS fix, decimal degrees.
|-
|altitude||int4||YES||Altitude above sea level measured by GPS tag in meters
|-
|pressure||int4||YES||Pressure measured by GPS tag sensor in Pascals
|-
|temperature||float8||YES||Temperature measured by GPS tag sensor in Celsius
|-
|h_accuracy||float8||YES||Horizontal accuracy (meters)
|-
|v_accuracy||float8||YES||vertical accuracy (meters)
|-
|x_speed||float8||YES||meters/second
|-
|y_speed||float8||YES||meters/second
|-
|z_speed||float8||YES||meters/second
|-
|gps_fixtime||float8||YES||GPS fix time (also known as Time to Fix) in seconds
|-
|userflag||int4||YES||Acceptable = 0; Data flagged as unacceptable by user if not equal to 0.
|-
|satellites_used||int2||YES||Number of satellites used for fix the GPS measurement
|-
|positiondop||float8||YES||Positional DOP (Dilution of Precision), unitless
|-
|speed_accuracy||float8||YES||
|-
|location||geometry||YES||postgreSQL geometry; to allow GIS operations
|-
|vnorth||float8||YES||- vx * sin(lat) * cos(long) - vy * sin(lat) * sin (long) + vz * cos(lat), m/s
|-
|veast||float8||YES||- vx * sin(lon) + vy * cos(lon), m/s
|-
|vdown||float8||YES||- vx * cos(lat) * cos(lon) - vy * cos(lat) * sin(lon) - vz * sin(lat), m/s
|-
|speed||float8||YES||- vx * sin(lon) + vy * cos(lon)
|-
|speed_3d||float8||YES|| sqrt(d.x_speed^2 + d.y_speed^2 + d.z_speed^2), m/s
|-
|speed_2d||float8||YES|| Speed on 2D, m/s
|-
|direction||numeric||YES|| Heading in degrees; 0 is north, 90 is east
|-
|altitude_agl||float8||YES|| Reported GPS altitude MINUS terrain elevation on the coordinate based on Digital Elevation Model data used in the e-ecology database (for further information see: [[https://services.e-ecology.sara.nl/wiki/index.php/Elevation]]
|}

=== ''gps.''ee_acceleration_limited ===

Each row represents an acceleration measurement as it is interpreted from the data that comes via the accelerometer. Acceleration is looked at in batches for a period of time, for 3 components (X,Y,Z). The blocks are delimited by a starting date and time, and then each observation is numbered consecutively.

The logger tries to mark the start of blocks from the GPS fix that coincides with the beginning of the acceleration block. If there's no GPS fix during that block, then it extrapolates the date and time and makes a special record which is interpreted by our accelerometer processing software and you can find it via gps.ee_acc_start_limited. Therefore, to know all valid dates and times that mark a valid block for a tracker, you need to consider all of the values in column date_time on gps.ee_tracking_speed_limited for the given tracker, column date_time on gps.ee_acc_start_limited, and all the values on column date_time for that tracker. And the order is important within each block, so you should sort the acceleration measurements by block and index.

This table structure has ''not'' changed with respect to our previous database.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker that reported the fix. Part of the unique identifier of the GPS fix.
|-
|date_time||timestamp||NO||Time of the acceleration block that this measurement belongs to.
|-
|index||int2||NO||Sorting order of the measurement within the acceleration block (each block starts at 0).
|-
|x_acceleration||int2||YES||Acceleration measured on the x (or surge) axis (range between -2666 and 2666).
|-
|y_acceleration||int2||YES||Acceleration measured on the y (or sway) axis (range between -2666 and 2666).
|-
|z_acceleration||int2||YES||Acceleration measured on the z (or heave) axis (range between -2666 and 2666).
|}
=== ''gps.''ee_acc_start_limited ===

Each row represents the start of a block of acceleration measurements when a GPS fix is not available at that acceleration block's time.

The logger tries to mark the start of blocks from the GPS fix that coincides with the begin of the acceleration block. If there's no GPS fix during that block, then it extrapolates the date and time and makes a special record which is interpreted by our accelerometer processing software and made available through this view.

Therefore, to know all valid dates and times that mark a valid block for a tracker, you need to consider all of the values in column date_time on gps.ee_tracking_speed_limited for the given tracker, column date_time on gps.ee_acc_start_limited, and all the values on column date_time for that tracker. And the order is important within each block, so you should sort the acceleration measurements by block and index.

Added in firmware 1.0.2.7.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker that reported the acceleration measurement.
|-
|date_time||timestamp||NO||Time to mark the start of the acceleration block.
|-
|line_counter||int4||NO||Number of lines associated with this set of accelerations.
|-
|timesynced||int2||YES||The source of the time that marks the acceleration block (0=not synced, 1=user set, 2=Gps Synced).
|-
|ascii||int4||YES||Acceleration measurement interval.
|-
|accsn||int4||YES||Number of samples (0-65536).
|-
|f||int2||YES||Acceleration sampling frequency (0=20Hz, 1=10Hz, 2=5Hz, 3=1Hz).
|}

== Relation to projects ==

=== ''gps.''ee_user_limited ===

Each row shows a colleague in a project that you belong to, indicating her role and status there.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||Project name
|-
|email||varchar||NO||e-mail address
|-
|login_name||varchar||NO||Login name
|-
|name||varchar||NO||(first) name
|-
|surname||varchar||NO||Surname
|-
|role||varchar||NO||Role in the project
|-
|status||varchar||NO||Status of the membership
|-
|user_id||int8||NO||Unique identifier of the user. Included to enable joins.
|}

=== ''gps.''ee_tracker_ownership_limited ===

Each row that you see expresses that a tracker belongs to one of the projects you are involved with.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||The project that the tracker belongs to.
|-
|device_info_serial||int8||NO||Serial number of the tracker.
|-
|project_id||int8||NO||Reference to the project. Included just to enable joins.
|-
|tracker_id||int8||NO||Reference to the tracker. Included just to enable joins.
|}

=== ''gps.''ee_individual_ownership_limited ===

Each row that you see expresses that a bird belongs to one of the projects you are involved with.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||The project that the individual belongs to.
|-
|ring_number||varchar||NO||Ring number of the bird.
|-
|project_id||int8||NO||Reference to the project. Included just to enable joins.
|-
|individual_id||int8||NO||Reference to the individual. Included just to enable joins.
|}

== Engineering data ==

'''Note:''' These views will show no rows for trackers not yet attached to birds.

=== ''gps.''ee_comm_limited ===

Each row indicates status and result of communications with ground stations for each tracker that your project allows you to see.

Only for newer firmware (v.1.0.1.4 and higher).

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
| device_info_serial || int4 || NO || Serial number of the tracker.
|-
| date_time || timestamp || NO || timestamp of start communication
|-
| date_time_end || timestamp ||YES|| timestamp of end communication
|-
| date_time_utc || timestamp ||YES|| date_time in utc (if available)
|-
| date_time_end_utc || timestamp ||YES|| date_time_end in utc (if available)
|-
| action || text||YES|| JOINED / RESCHEDULE ??
|-
| bytes_communicated || int4 ||YES|| Number of bytes communicated (total of all bytes per DATA line)
|-
|status || text ||YES|| READY, TIMEOUT, LOWPOWER, MISSING
|-
| network_join_time || int4||YES||milliseconds
|-
| number_identical_lines || int4 ||YES|| Number identical lines in data block. Gives indication
|-
| non_ascii_char || int4 ||YES|| Number of non ascii characters (indicated by question-mark ?)
|-
| sts_time || timestamp ||YES|| derived from S0 hh,mi
|-
| timesynced || int2||YES|| TimeSynced (0=not sy:wnced, 1=user set, 2=Gps Synced)
|-
|devicemode || int2||YES||DeviceMode (0-99). This describes whether the device is in a special operational mode. 0 = normal operational mode; 80 hex = Memory save mode (memory is almost full and all activity is drastically reduced so the circular memory probably won't be overwritten or, if it does, it will be kept to a minimum)
|-
| vbat || float8 ||YES|| Battery voltage in V (0-~4.200V)
|-
|logb || int4 ||YES|| logged bytes, waiting for download (0-4Mbyte)
|-
| logp || int2 ||YES|| Logged percentage (units 0.1%) of memory waiting to be downloaded (0-1000)
|-
| gpsii || int4 ||YES|| GpsIval, current (0-65535)
|-
|comii || int4||YES|| ComIval, current (0-65535)
|-
|accii || int4||YES|| AcceleroIval, current (0-65535)
|-
|swddd || int4||YES|| User switches, current (0-65535)
|-
|last_gps_fix ||timestamp||YES||timestamp of last fix
|-
|latitude||float8||YES||Latitude of last fix. Decimal degrees
|-
|longitude||float8||YES||Longitude of last fix. Decimal degrees
|}

=== ''gps.''ee_energy_limited ===

Only for newer firmware.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker
|-
|date_time||timestamp||NO||Date and time of the reported data
|-
|timesynced ||int2||YES||TimeSynced (0=not synced, 1=user set, 2=Gps synced)
|-
|devicemode ||int2||YES||(0-99)
|-
|vsll ||float8||YES||Solar Voltage during charging in mV (0-~2500)
|-
|vbat ||float8||YES||Battery Voltage in V (0-~4.200)
|-
|ssw ||int2||YES|| Solar Cell Switches ... (0-255) fw1022 and up
|-
| ||||YES||extended fields if special user switch is on
|-
|vsoo ||float8||YES|| Solar voltage open in V (0-~2.500)
|-
|vdrf ||float8||YES||reference voltage Dac in V (0-~2.435)
|-
|vchr ||float8||YES||charge current voltage
|-
|g ||int2||YES|| Gps battery threshold OK
|-
|c || int2||YES||Com battery threshold OK
|-
|m || int2||YES||Charge Mode (0,1,2 Com battery threshold OK
|-
|pressure || int4||YES|| in pascal (1mBar = 100Pa, 1pressuremeter = ~12Pa)
|-
|temperature || float8||YES|| in celcius degrees
|}

=== ''gps.''ee_settings_limited ===

Settings are either from C0-C1-C9 lines or parsed from uplink communication lines.
If C0-C1 is parsed its values are compared with the previous value and if the settings are the same, then no new records need to be added.
If uplink settings are parsed, then wait until the last setting has been seen (DL command or EOF); then store all the new settings in one record. Only for newer firmware.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||
|-
|date_time||timestamp||NO||Derived timestamp from Hh:mi fields in C0 combined with computer timestamp in JOINED line
|-
|timesynced || int2 ||YES||
|-
|timezone_time || int2 ||YES||
|-
|swddd || int4||YES|| SpecialUserSwitches
|-
|gpsnavmode || int2||YES|| GpsNavMode 0-8. This is the Kalman filter used for GPS navigation mode often called the "Dynamic platform model". The following codes are provided by the GPS chip manufacturer for the UBlox GPS chip LEAS4S: 1 = stationary, 2 = pedestrian, 3 = automotive, 4 = sea, 5 = airborne 1G, 6 = airborne 2G, 7 = airborne 3G. The UBlox GPS chip LEAS6S uses the following codes: 0 = portable, 2 = stationary, 3 = pedestrian, 4 = automotive, 5 = sea, 6 = airborne with < 1g acceleration, 7 = airborne with < 2g acceleration, 8 = airborne with < 4g acceleration
|-
|gpsreceivermode || int2 ||YES|| GpsReceiverNode 0-3
|-
|gpsfixtime || int2||YES|| GpsFixTime 0-255
|-
|gps_ivaln_on_h || int2||YES|| hours Gps IvalN on
|-
|gps_ivaln_on_m || int2||YES|| minutes Gps IvalN on
|-
|gps_ivaln || int4||YES|| GpsIvalN (0-65530)
|-
|gps_ivale_on_h || int2||YES|| hours Gps IvalE on
|-
|gps_ivale_on_m || int2||YES|| minutes GpS IvalE on
|-
|gps_ivale || int4||YES|| GpsIvalE
|-
|com_ivaln_on_h || int2||YES|| hours Com IvalN on
|-
|com_ivaln_on_m || int2||YES|| minutes Com IvalN on
|-
|com_ivaln || int4||YES|| ComIvalN (0-65530)
|-
|com_ivale_on_h || int2||YES|| hours Com IvalE on
|-
|com_ivale_on_m || int2||YES|| minutes Com IvalE on
|-
|com_ivale || int4||YES|| ComIvalE
|-
|accii || int4||YES|| Accelero measurement interval
|-
|accbn || int4||YES|| Accelero number of blocks
|-
|accsn || int4||YES|| Accelero number of samples (0-65536)
|-
|f || int2||YES||Accelero sample frequency (0=20Hz, 1=10Hz, 2=5Hz, 3=1Hz)
|-
|pdp || int2||YES|| PdopMsk (3-250)
|-
|pac||int2||YES|| PaccMsk (10-100)
|-
|tdp ||int2||YES|| TdopMsk (3-250)
|-
|tac ||int2||YES|| TaccMsk (30-300)
|-
|ga || int2||YES|| Additional Fixes GpsIval mode (3-25) before interval fix is logged
|-
|aol || int2||YES||AccMemOffLev (100 - 950) units 0.1%; if Memfull above this level all Accelero activity will be overruled and switched off untill MemFull < level - 5
|-
|eventnum || text||YES|| 12 eventnumbers (0-0xff) only printed if not zero, therfore converted to text
|}

=== ''gps.''ee_trackingfile_limited ===

Each row shows the status of the last attempt to parse (process) the raw data of a file for a given tracker, for each tracker that your projects allow you to see.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|id||int8||NO||Unique identifier of the row
|-
|date_reported||date||NO||Date that the file arrived
|-
|filename||varchar||NO||Name of the file
|-
|last_modified_date||timestamp||NO||Date that the file was last modified on
|-
|device_info_serial||int4||YES||Serial number of the tracker this file comes from
|-
|filesize||int8||NO||Size of the file in KB
|-
|parsed_date||timestamp||YES||Date and time that the file was last attempted to be parsed on
|-
|version||int4||NO||Optimistic locking field. Ignore it as a user.
|-
|last_found_date||timestamp||YES||Last time that a process that runs to look for new files saw the file
|-
|n_lines||int4||YES||Count of lines found in the file last time it was processed
|-
|n_data_lines||int4||YES||Count of actual data lines, out of the total lines
|-
|n_tracking_records||int4||YES||Count of GPS fixes found during the parsing process
|-
|n_communication_records||int4||YES||Count of communication records found during the parsing process
|-
|n_energy_records||int4||YES||Count of energy records found during the parsing process
|-
|n_acceleration_records||int4||YES||Count of acceleration records found during the parsing process
|-
|n_acceleration_start_records||int4||YES||Count of acceleration start records found during the parsing process
|-
|n_errors||int4||YES||Count of errors encountered during the parsing process
|-
|n_settings_records||int4||YES||Count of settings records found during the parsing process
|-
|filepath||text||NO||Directory path of the file on the server
|}

=== ''gps.''ee_warning_limited ===

Since firmware version 1.0.2.7 this table was added for events that do not occur very often (C10, C20) or errors that occur during parsing.

For example all CRC failures can be selected with :

select * from gps.ee_warning_limited
where warning = 'CRC8 failure'
order by device_info_serial;

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker
|-
|filename||text||NO||File name
|-
|linenumber||integer||YES||line number in file
|-
|warning||text||YES||The warning message
|-
|line||text||YES||The actual line contents in the file
|}

== Sharings ==

Projects can now share trackers with other projects during periods of time. This means that people in the receiving projects can access information that these shared trackers provide during those time periods.

The following views allow you to see this shared information:

* ''Leaf'' data
** ''gps.''ee_shared_tracker_limited: information about each shared tracker
** ''gps.''ee_shared_individual_limited: information about birds with shared trackers
** ''gps.''ee_shared_track_session_limited: information about track sessions which involve shared trackers
* Tracking data
** ''gps.''ee_shared_tracking_speed_limited: tracking information (gps, speed), during the shared period, for trackers shared
** ''gps.''ee_shared_acc_start_limited: acceleration start information, during the shared period, for trackers shared
** ''gps.''ee_shared_acceleration_limited: acceleration information for trackers shared
* Relation to projects
** ''gps.''ee_shared_tracker_sharing_limited: view information for each sharing that your projects allow you to see

UvA-BiTS Tracking Data

2014-12-09T13:41:33Z

Judy: /* gps.ee_tracking_speed_limited */

== ''Meta'' data ==

=== ''gps.''ee_project_limited ===

Each row shows the details of each project, for the projects you are allowed to see.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||Unique key name, with the form <species>_<place>
|-
|station_name||varchar||NO||Where the receiving station is
|-
|start_date||timestamp||NO||Project start date
|-
|end_date||timestamp||NO||Date at which the project is not relevant any more
|-
|description||text||YES||String for free description. Updatable by the user.
|-
|project_id||int8||NO||Project unique id. Included just to enable joins.
|-
|parent_id||int8||YES||Reference to the parent project in the hierarchy.
|}

=== ''gps.''ee_tracker_limited ===

Each row shows tracker details for all trackers that your projects allow you to see.

This view replaces the old gps.uva_device_limited.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int8||NO||Serial number of the tracker
|-
|firmware_version||varchar||YES||String for firmware version
|-
|mass||numeric(4,2)||YES||Mass (in grams)
|-
|start_date||timestamp||NO||Date before which the device is not relevant
|-
|end_date||timestamp||NO||Date at which the device is not relevant any more
|-
|x_o||numeric(30,6)||YES||Accelerometer calibration: offset on the x (or surge) axis
|-
|x_s||numeric(30,6)||YES||Accelerometer calibration: sensitivity on the x (or surge) axis
|-
|y_o||numeric(30,6)||YES||Accelerometer calibration: offset on the y (or sway) axis
|-
|y_s||numeric(30,6)||YES||Accelerometer calibration: sensitivity on the y (or sway) axis
|-
|z_o||numeric(30,6)||YES||Accelerometer calibration: offset on the z (or heave) axis
|-
|z_s||numeric(30,6)||YES||Accelerometer calibration: sensitivity on the y (or heave) axis
|-
|tracker_id||int8||NO||Tracker unique id. Included just to enable joins.
|}

=== ''gps.''ee_individual_limited ===

Each row shows bird's details for all birds that your projects allow you to see.

This view replaces the old gps.uva_individual_limited.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|ring_number||varchar||NO||Ring number.
|-
|species_latin_name||varchar||NO||Latin name of the species of the bird.
|-
|colour_ring||varchar||YES||String for ring colour.
|-
|sex||varchar||NO||Sex (M=male, F=female, X=unknown/unspecified)
|-
|mass||numeric(5,0)||YES||Mass (grams)
|-
|start_date||timestamp||NO||Date when the bird was first tracked with an UvA-BiTS logger.
|-
|end_date||timestamp||NO||Date at which the bird is no longer relevant for the tracking project.
|-
|remarks||text||YES||Text for user remarks
|-
|individual_id||int8||NO||Bird unique id. Included just to enable joins.
|}

=== ''gps.''ee_nest_limited ===
This view shows nests you have access to. A nest is always for a bird with a tracker (so a track session must exist for that bird), and must contain a short name (a reference name for you to use) and a latitude and longitude pair.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|reference_name||varchar||NO||A string with a useful reference for you. We recommend its being unique for each nest in a project, but the system does NOT enforce it.
|-
|latitude||numeric||YES||Latitude of the coordinates where the nest was found.
|-
|longitude||numeric||YES||Longitude of the coordinates where the nest was found.
|-
|start_date_time||timestamp||NO||When the nest was spotted.
|-
|end_date_time||timestamp||NO||When the nest ceases to be relevant.
|-
|found_by_whom||varchar||YES||Place to credit people involved in finding the nest.
|-
|remarks||text||YES||Free text.
|-
|location||bytearray||YES||Calculated Geometry of the coordinates where the nest was found.
|-
|nest_id||int8||NO||Unique id. Added to allow joins.
|}

=== ''gps.''ee_nest_inhabitant_limited ===
This view shows who lives on which nest, for birds you have access to.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|reference_name||varchar||NO||A string belonging to the nest that you can use to identify that nest. Because it is NOT enforced to be unique, DO NOT USE THIS FIELD FOR JOINING.
|-
|ring_number||varchar||NO||The ring number of a bird on a nest.
|-
|device_info_serial||int8||NO||Following the link from a bird to her track sessions, and considering the nest's start and end time, the bird's trackers within that period are resolved so that they can be shown here. This is the device_info_serial of a tracker.
|-
|key_name||varchar||NO||Following the link from a bird to her track sessions, and considering the nest's start and end time, the track sessions' projects within that period are resolved so that they can be shown here. This is the key_name of a project.
|-
|nest_id||int8||NO||Reference to the nest id. Added to allow joins.
|-
|individual_id||int8||NO||Reference to the individual id. Added to allow joins.
|-
|track_session_id||int8||NO||Reference to a resolved track session id. Added to allow joins.
|}

=== ''gps.''ee_track_session_limited ===

A track session links a tracker and a bird for a given period of time. Therefore, it is '''mandatory for projects''' to supply this piece of '''metadata''' before they can see any tracking data related to a tracker they own.

In order to create a track session, the project must create first an entry in the system for the bird that they are attaching the tracker to, and this is the way for the UvA-BiTS community to make sense of the tracking data that we are receiving. This is why providing the metadata is so important.

By editing the start and end dates, administrators of the project that this track session belongs to, can delimit (and therefore, establish) when the coupling makes sense (or is to be understood as useful).

Whenever a tracker is attached to a bird, a track session must be started (start_date) to indicate that the tracking data that is received is already live data. If the tracker is separated from the bird, the tracking data ceases to make sense, so the track session's end date should be set to denote it has ended. New track sessions can be started if the tracker is re-attached to a bird (possibly the same as before, or maybe a different one) afterwards.

Each row shows track session's information. There exists a row for each track session that your projects allow you to see.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||The project that the track session belongs to.
|-
|device_info_serial||int8||NO||Serial number of the tracker this track session involves.
|-
|ring_number||varchar||NO||Ring number of the bird that this track session involves.
|-
|start_date||timestamp||NO||Date before which the track session is not relevant
|-
|end_date||timestamp||NO||Date at which the track session is not relevant any more
|-
|remarks||text||YES||Free text for comments
|-
|start_latitude||numeric(11,8)||NO||Latitude where the bird was tagged
|-
|start_longitude||numeric(11,8)||NO||Longitude where the bird was tagged
|-
|project_id||int8||NO||Reference to the project this track session belongs to. Included just to enable joins.
|-
|tracker_id||int8||NO||Reference to the tracker this track session involves. Included just to enable joins.
|-
|individual_id||int8||NO||Reference to the individual this track session involves. Included just to enable joins.
|-
|track_session_id||int8||NO||Unique identifier of the track session. Included just to enable joins.
|}

== Tracking data ==

'''Note:''' These views will show no rows for trackers not yet attached to birds.

=== ''gps.''ee_tracking_speed_limited ===

Shows the tracking data that your projects allow you to see.

Each row represents a GPS fix as it is interpreted from the data that comes via the GPS tracker, plus it adds speeds.

It replaces old views gps.uva_tracking_limited, gps.uva_tracking_speed_3d_limited and gps.uva_tracking_speed. This table structure has ''not'' changed with respect to our previous database.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker that reported the fix. Part of the unique identifier of the GPS fix.
|-
|date_time||timestamp||NO||Time when the fix was taken. UTC (i.e.: no timezone). Part of the unique identifier of the GPS fix.
|-
|latitude||float8||YES||Latitude of the GPS fix, decimal degrees.
|-
|longitude||float8||YES||Longitude of the GPS fix, decimal degrees.
|-
|altitude||int4||YES||Altitude above sea level measured by GPS tag in meters
|-
|pressure||int4||YES||Pressure measured by GPS tag sensor in Pascals
|-
|temperature||float8||YES||Temperature measured by GPS tag sensor in Celsius
|-
|h_accuracy||float8||YES||Horizontal accuracy (meters)
|-
|v_accuracy||float8||YES||vertical accuracy (meters)
|-
|x_speed||float8||YES||meters/second
|-
|y_speed||float8||YES||meters/second
|-
|z_speed||float8||YES||meters/second
|-
|gps_fixtime||float8||YES||GPS fix time (also known as Time to Fix) in seconds
|-
|userflag||int4||YES||Acceptable = 0; Data flagged as unacceptable by user if not equal to 0.
|-
|satellites_used||int2||YES||Number of satellites used for fix the GPS measurement
|-
|positiondop||float8||YES||Positional DOP (Dilution of Precision), unitless
|-
|speed_accuracy||float8||YES||
|-
|location||geometry||YES||postgreSQL geometry; to allow GIS operations
|-
|vnorth||float8||YES||- vx * sin(lat) * cos(long) - vy * sin(lat) * sin (long) + vz * cos(lat), m/s
|-
|veast||float8||YES||- vx * sin(lon) + vy * cos(lon), m/s
|-
|vdown||float8||YES||- vx * cos(lat) * cos(lon) - vy * cos(lat) * sin(lon) - vz * sin(lat), m/s
|-
|speed||float8||YES||- vx * sin(lon) + vy * cos(lon)
|-
|speed_3d||float8||YES|| sqrt(d.x_speed^2 + d.y_speed^2 + d.z_speed^2), m/s
|-
|speed_2d||float8||YES|| Speed on 2D, m/s
|-
|direction||numeric||YES|| Heading in degrees; 0 is north, 90 is east
|-
|altitude_agl||float8||YES|| Reported GPS altitude MINUS terrain elevation on the coordinate based on Digital Elevation Model data used in the e-ecology database (for further information see: [[https://services.e-ecology.sara.nl/wiki/index.php/Elevation]]
|}

=== ''gps.''ee_acceleration_limited ===

Each row represents an acceleration measurement as it is interpreted from the data that comes via the accelerometer. Acceleration is looked at in batches for a period of time, for 3 components (X,Y,Z). The blocks are delimited by a starting date and time, and then each observation is numbered consecutively.

The logger tries to mark the start of blocks from the GPS fix that coincides with the beginning of the acceleration block. If there's no GPS fix during that block, then it extrapolates the date and time and makes a special record which is interpreted by our accelerometer processing software and you can find it via gps.ee_acc_start_limited. Therefore, to know all valid dates and times that mark a valid block for a tracker, you need to consider all of the values in column date_time on gps.ee_tracking_speed_limited for the given tracker, column date_time on gps.ee_acc_start_limited, and all the values on column date_time for that tracker. And the order is important within each block, so you should sort the acceleration measurements by block and index.

This table structure has ''not'' changed with respect to our previous database.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker that reported the fix. Part of the unique identifier of the GPS fix.
|-
|date_time||timestamp||NO||Time of the acceleration block that this measurement belongs to.
|-
|index||int2||NO||Sorting order of the measurement within the acceleration block (each block starts at 0).
|-
|x_acceleration||int2||YES||Acceleration measured on the x (or surge) axis (range between -2666 and 2666).
|-
|y_acceleration||int2||YES||Acceleration measured on the y (or sway) axis (range between -2666 and 2666).
|-
|z_acceleration||int2||YES||Acceleration measured on the z (or heave) axis (range between -2666 and 2666).
|}
=== ''gps.''ee_acc_start_limited ===

Each row represents the start of a block of acceleration measurements when a GPS fix is not available at that acceleration block's time.

The logger tries to mark the start of blocks from the GPS fix that coincides with the begin of the acceleration block. If there's no GPS fix during that block, then it extrapolates the date and time and makes a special record which is interpreted by our accelerometer processing software and made available through this view.

Therefore, to know all valid dates and times that mark a valid block for a tracker, you need to consider all of the values in column date_time on gps.ee_tracking_speed_limited for the given tracker, column date_time on gps.ee_acc_start_limited, and all the values on column date_time for that tracker. And the order is important within each block, so you should sort the acceleration measurements by block and index.

Added in firmware 1.0.2.7.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker that reported the acceleration measurement.
|-
|date_time||timestamp||NO||Time to mark the start of the acceleration block.
|-
|line_counter||int4||NO||Number of lines associated with this set of accelerations.
|-
|timesynced||int2||YES||The source of the time that marks the acceleration block (0=not synced, 1=user set, 2=Gps Synced).
|-
|ascii||int4||YES||Acceleration measurement interval.
|-
|accsn||int4||YES||Number of samples (0-65536).
|-
|f||int2||YES||Acceleration sampling frequency (0=20Hz, 1=10Hz, 2=5Hz, 3=1Hz).
|}

== Relation to projects ==

=== ''gps.''ee_user_limited ===

Each row shows a colleague in a project that you belong to, indicating her role and status there.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||Project name
|-
|email||varchar||NO||e-mail address
|-
|login_name||varchar||NO||Login name
|-
|name||varchar||NO||(first) name
|-
|surname||varchar||NO||Surname
|-
|role||varchar||NO||Role in the project
|-
|status||varchar||NO||Status of the membership
|-
|user_id||int8||NO||Unique identifier of the user. Included to enable joins.
|}

=== ''gps.''ee_tracker_ownership_limited ===

Each row that you see expresses that a tracker belongs to one of the projects you are involved with.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||The project that the tracker belongs to.
|-
|device_info_serial||int8||NO||Serial number of the tracker.
|-
|project_id||int8||NO||Reference to the project. Included just to enable joins.
|-
|tracker_id||int8||NO||Reference to the tracker. Included just to enable joins.
|}

=== ''gps.''ee_individual_ownership_limited ===

Each row that you see expresses that a bird belongs to one of the projects you are involved with.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|key_name||varchar||NO||The project that the individual belongs to.
|-
|ring_number||varchar||NO||Ring number of the bird.
|-
|project_id||int8||NO||Reference to the project. Included just to enable joins.
|-
|individual_id||int8||NO||Reference to the individual. Included just to enable joins.
|}

== Engineering data ==

'''Note:''' These views will show no rows for trackers not yet attached to birds.

=== ''gps.''ee_comm_limited ===

Each row indicates status and result of communications with ground stations for each tracker that your project allows you to see.

Only for newer firmware (v.1.0.1.4 and higher).

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
| device_info_serial || int4 || NO || Serial number of the tracker.
|-
| date_time || timestamp || NO || timestamp of start communication
|-
| date_time_end || timestamp ||YES|| timestamp of end communication
|-
| date_time_utc || timestamp ||YES|| date_time in utc (if available)
|-
| date_time_end_utc || timestamp ||YES|| date_time_end in utc (if available)
|-
| action || text||YES|| JOINED / RESCHEDULE ??
|-
| bytes_communicated || int4 ||YES|| Number of bytes communicated (total of all bytes per DATA line)
|-
|status || text ||YES|| READY, TIMEOUT, LOWPOWER, MISSING
|-
| network_join_time || int4||YES||milliseconds
|-
| number_identical_lines || int4 ||YES|| Number identical lines in data block. Gives indication
|-
| non_ascii_char || int4 ||YES|| Number of non ascii characters (indicated by question-mark ?)
|-
| sts_time || timestamp ||YES|| derived from S0 hh,mi
|-
| timesynced || int2||YES|| TimeSynced (0=not sy:wnced, 1=user set, 2=Gps Synced)
|-
|devicemode || int2||YES||DeviceMode (0-99). This describes whether the device is in a special operational mode. 0 = normal operational mode; 80 hex = Memory save mode (memory is almost full and all activity is drastically reduced so the circular memory probably won't be overwritten or, if it does, it will be kept to a minimum)
|-
| vbat || float8 ||YES|| Battery voltage in V (0-~4.200V)
|-
|logb || int4 ||YES|| logged bytes, waiting for download (0-4Mbyte)
|-
| logp || int2 ||YES|| Logged percentage (units 0.1%) of memory waiting to be downloaded (0-1000)
|-
| gpsii || int4 ||YES|| GpsIval, current (0-65535)
|-
|comii || int4||YES|| ComIval, current (0-65535)
|-
|accii || int4||YES|| AcceleroIval, current (0-65535)
|-
|swddd || int4||YES|| User switches, current (0-65535)
|-
|last_gps_fix ||timestamp||YES||timestamp of last fix
|-
|latitude||float8||YES||Latitude of last fix. Decimal degrees
|-
|longitude||float8||YES||Longitude of last fix. Decimal degrees
|}

=== ''gps.''ee_energy_limited ===

Only for newer firmware.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker
|-
|date_time||timestamp||NO||Date and time of the reported data
|-
|timesynced ||int2||YES||TimeSynced (0=not synced, 1=user set, 2=Gps synced)
|-
|devicemode ||int2||YES||(0-99)
|-
|vsll ||float8||YES||Solar Voltage during charging in mV (0-~2500)
|-
|vbat ||float8||YES||Battery Voltage in V (0-~4.200)
|-
|ssw ||int2||YES|| Solar Cell Switches ... (0-255) fw1022 and up
|-
| ||||YES||extended fields if special user switch is on
|-
|vsoo ||float8||YES|| Solar voltage open in V (0-~2.500)
|-
|vdrf ||float8||YES||reference voltage Dac in V (0-~2.435)
|-
|vchr ||float8||YES||charge current voltage
|-
|g ||int2||YES|| Gps battery threshold OK
|-
|c || int2||YES||Com battery threshold OK
|-
|m || int2||YES||Charge Mode (0,1,2 Com battery threshold OK
|-
|pressure || int4||YES|| in pascal (1mBar = 100Pa, 1pressuremeter = ~12Pa)
|-
|temperature || float8||YES|| in celcius degrees
|}

=== ''gps.''ee_settings_limited ===

Settings are either from C0-C1-C9 lines or parsed from uplink communication lines.
If C0-C1 is parsed its values are compared with the previous value and if the settings are the same, then no new records need to be added.
If uplink settings are parsed, then wait until the last setting has been seen (DL command or EOF); then store all the new settings in one record. Only for newer firmware.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||
|-
|date_time||timestamp||NO||Derived timestamp from Hh:mi fields in C0 combined with computer timestamp in JOINED line
|-
|timesynced || int2 ||YES||
|-
|timezone_time || int2 ||YES||
|-
|swddd || int4||YES|| SpecialUserSwitches
|-
|gpsnavmode || int2||YES|| GpsNavMode 0-8. This is the Kalman filter used for GPS navigation mode often called the "Dynamic platform model". The following codes are provided by the GPS chip manufacturer for the UBlox GPS chip LEAS4S: 1 = stationary, 2 = pedestrian, 3 = automotive, 4 = sea, 5 = airborne 1G, 6 = airborne 2G, 7 = airborne 3G. The UBlox GPS chip LEAS6S uses the following codes: 0 = portable, 2 = stationary, 3 = pedestrian, 4 = automotive, 5 = sea, 6 = airborne with < 1g acceleration, 7 = airborne with < 2g acceleration, 8 = airborne with < 4g acceleration
|-
|gpsreceivermode || int2 ||YES|| GpsReceiverNode 0-3
|-
|gpsfixtime || int2||YES|| GpsFixTime 0-255
|-
|gps_ivaln_on_h || int2||YES|| hours Gps IvalN on
|-
|gps_ivaln_on_m || int2||YES|| minutes Gps IvalN on
|-
|gps_ivaln || int4||YES|| GpsIvalN (0-65530)
|-
|gps_ivale_on_h || int2||YES|| hours Gps IvalE on
|-
|gps_ivale_on_m || int2||YES|| minutes GpS IvalE on
|-
|gps_ivale || int4||YES|| GpsIvalE
|-
|com_ivaln_on_h || int2||YES|| hours Com IvalN on
|-
|com_ivaln_on_m || int2||YES|| minutes Com IvalN on
|-
|com_ivaln || int4||YES|| ComIvalN (0-65530)
|-
|com_ivale_on_h || int2||YES|| hours Com IvalE on
|-
|com_ivale_on_m || int2||YES|| minutes Com IvalE on
|-
|com_ivale || int4||YES|| ComIvalE
|-
|accii || int4||YES|| Accelero measurement interval
|-
|accbn || int4||YES|| Accelero number of blocks
|-
|accsn || int4||YES|| Accelero number of samples (0-65536)
|-
|f || int2||YES||Accelero sample frequency (0=20Hz, 1=10Hz, 2=5Hz, 3=1Hz)
|-
|pdp || int2||YES|| PdopMsk (3-250)
|-
|pac||int2||YES|| PaccMsk (10-100)
|-
|tdp ||int2||YES|| TdopMsk (3-250)
|-
|tac ||int2||YES|| TaccMsk (30-300)
|-
|ga || int2||YES|| Additional Fixes GpsIval mode (3-25) before interval fix is logged
|-
|aol || int2||YES||AccMemOffLev (100 - 950) units 0.1%; if Memfull above this level all Accelero activity will be overruled and switched off untill MemFull < level - 5
|-
|eventnum || text||YES|| 12 eventnumbers (0-0xff) only printed if not zero, therfore converted to text
|}

=== ''gps.''ee_trackingfile_limited ===

Each row shows the status of the last attempt to parse (process) the raw data of a file for a given tracker, for each tracker that your projects allow you to see.

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|id||int8||NO||Unique identifier of the row
|-
|date_reported||date||NO||Date that the file arrived
|-
|filename||varchar||NO||Name of the file
|-
|last_modified_date||timestamp||NO||Date that the file was last modified on
|-
|device_info_serial||int4||YES||Serial number of the tracker this file comes from
|-
|filesize||int8||NO||Size of the file in KB
|-
|parsed_date||timestamp||YES||Date and time that the file was last attempted to be parsed on
|-
|version||int4||NO||Optimistic locking field. Ignore it as a user.
|-
|last_found_date||timestamp||YES||Last time that a process that runs to look for new files saw the file
|-
|n_lines||int4||YES||Count of lines found in the file last time it was processed
|-
|n_data_lines||int4||YES||Count of actual data lines, out of the total lines
|-
|n_tracking_records||int4||YES||Count of GPS fixes found during the parsing process
|-
|n_communication_records||int4||YES||Count of communication records found during the parsing process
|-
|n_energy_records||int4||YES||Count of energy records found during the parsing process
|-
|n_acceleration_records||int4||YES||Count of acceleration records found during the parsing process
|-
|n_acceleration_start_records||int4||YES||Count of acceleration start records found during the parsing process
|-
|n_errors||int4||YES||Count of errors encountered during the parsing process
|-
|n_settings_records||int4||YES||Count of settings records found during the parsing process
|-
|filepath||text||NO||Directory path of the file on the server
|}

=== ''gps.''ee_warning_limited ===

Since firmware version 1.0.2.7 this table was added for events that do not occur very often (C10, C20) or errors that occur during parsing.

For example all CRC failures can be selected with :

select * from gps.ee_warning_limited
where warning = 'CRC8 failure'
order by device_info_serial;

{| class="wikitable"
! COLUMN_NAME
! TYPE_NAME
! IS_NULLABLE
! DESCRIPTION
|-
|device_info_serial||int4||NO||Serial number of the tracker
|-
|filename||text||NO||File name
|-
|linenumber||integer||YES||line number in file
|-
|warning||text||YES||The warning message
|-
|line||text||YES||The actual line contents in the file
|}

== Sharings ==

Projects can now share trackers with other projects during periods of time. This means that people in the receiving projects can access information that these shared trackers provide during those time periods.

The following views allow you to see this shared information:

* ''Leaf'' data
** ''gps.''ee_shared_tracker_limited: information about each shared tracker
** ''gps.''ee_shared_individual_limited: information about birds with shared trackers
** ''gps.''ee_shared_track_session_limited: information about track sessions which involve shared trackers
* Tracking data
** ''gps.''ee_shared_tracking_speed_limited: tracking information (gps, speed), during the shared period, for trackers shared
** ''gps.''ee_shared_acc_start_limited: acceleration start information, during the shared period, for trackers shared
** ''gps.''ee_shared_acceleration_limited: acceleration information for trackers shared
* Relation to projects
** ''gps.''ee_shared_tracker_sharing_limited: view information for each sharing that your projects allow you to see