Difference between revisions of "Argos Data Model"
(→Table MEASUREMENT) |
(→GPS Data Model) |
||
(46 intermediate revisions by 3 users not shown) | |||
Line 1: | Line 1: | ||
− | == GPS Data Model == | + | == Argos GPS Data Model == |
− | Last updated January | + | Last updated January 5, 2010 Bart Heupers - [[ UvAGPS ]] moved to special page |
− | |||
− | ''' | + | Updated January 22, 2008 Judy Shamoun-Baranes |
+ | |||
+ | '''NOTE:''' On July 2014 we moved all argos-related tables, views, functions and types to a new schema called '''argos'''. | ||
+ | |||
+ | '''e-ECOLOGY ARGOS TRACKING DATABASE''' | ||
The tracking database will include data from several species. Some species are trapped and then monitored during the breeding season while others are trapped during the winter season. Information is collected on each tracked bird, including biometric measurements. These measurements may be collected more than once. During the breeding season information is also collected about the gull breeding colony itself, location of nests, state of breeding (number of eggs and juveniles) and then juveniles themselves are marked and measured (although not tracked). | The tracking database will include data from several species. Some species are trapped and then monitored during the breeding season while others are trapped during the winter season. Information is collected on each tracked bird, including biometric measurements. These measurements may be collected more than once. During the breeding season information is also collected about the gull breeding colony itself, location of nests, state of breeding (number of eggs and juveniles) and then juveniles themselves are marked and measured (although not tracked). | ||
Line 20: | Line 23: | ||
Primary keys are noted by bold italic text. Foreign keys are noted by italic text in the Field column. | Primary keys are noted by bold italic text. Foreign keys are noted by italic text in the Field column. | ||
− | Microwave | + | Microwave will under no circumstances provide information about their checksum calculations. If feasible, SARA will duplicate the procedure and include a flag representing the microwave filtered data. |
'''TECHNICAL INFORMATION FROM TAGS''' | '''TECHNICAL INFORMATION FROM TAGS''' | ||
− | |||
== DEVICE_INFO == | == DEVICE_INFO == | ||
Line 32: | Line 34: | ||
|'''Field'''||'''Type'''||'''Key'''||'''Constraints'''||'''Comments''' | |'''Field'''||'''Type'''||'''Key'''||'''Constraints'''||'''Comments''' | ||
|- | |- | ||
− | + | |'''ptt_id'''||int4||PK||NOT NULL||serial number of PTT provided by Argos and used for communication with Argos satellite | |
− | |||
− | |'''ptt_id '''||int4||PK||NOT NULL||serial number of PTT provided by Argos and used for communication with Argos satellite | ||
|- | |- | ||
|serial ||int4||||NOT NULL||serial number of tag from manufacturer | |serial ||int4||||NOT NULL||serial number of tag from manufacturer | ||
Line 51: | Line 51: | ||
|- | |- | ||
|Remarks||text|||||| | |Remarks||text|||||| | ||
− | |||
|} | |} | ||
Line 61: | Line 60: | ||
|'''Field'''||'''Type'''||'''Key'''||'''Constraints'''||'''Comments''' | |'''Field'''||'''Type'''||'''Key'''||'''Constraints'''||'''Comments''' | ||
|- | |- | ||
− | |''id ''||int4 ||PK||NOT NULL||unique ID for each usage of a PTT so test and bird with same ptt_id are two separate IDs/indexes | + | |'''id '''||int4 ||PK||NOT NULL||unique ID for each usage of a PTT so test and bird with same ptt_id are two separate IDs/indexes |
|- | |- | ||
− | |'' | + | |''pttid''||int4 ||FK||NOT NULL||Link to DEVICE_INFO table |
|- | |- | ||
− | | | + | |''ring_number''||text ||FK||||link to individual table – tests do not have a ring_number |
|- | |- | ||
|start_date ||timestamptz||||||Date/time (Local) when tracking begins | |start_date ||timestamptz||||||Date/time (Local) when tracking begins | ||
Line 81: | Line 80: | ||
|time_steps ||text|||||| | |time_steps ||text|||||| | ||
|- | |- | ||
− | |remarks | + | |remarks||text|||||| |
− | |||
− | |||
|- | |- | ||
+ | |location ||geometry||||||postgreSQL geometry | ||
|} | |} | ||
− | |||
== GPS_TRACKING_DATA == | == GPS_TRACKING_DATA == | ||
Line 95: | Line 92: | ||
|'''Field'''||'''Type'''||'''Key'''||'''Constraints'''||'''Comments''' | |'''Field'''||'''Type'''||'''Key'''||'''Constraints'''||'''Comments''' | ||
|- | |- | ||
− | |'' | + | |'''ptt_id'''||int4||PK||NOT NULL || |
− | |||
− | |||
− | |||
− | |||
|- | |- | ||
− | |date_time||timestamp||||NOT NULL||GMT | + | |'''date_time'''||timestamp||PK||NOT NULL||GMT |
|- | |- | ||
|latitude||float8||||||Decimal degrees | |latitude||float8||||||Decimal degrees | ||
Line 121: | Line 114: | ||
|parser_QC||text||||||Data flagged as unacceptable by Microwave parser | |parser_QC||text||||||Data flagged as unacceptable by Microwave parser | ||
|- | |- | ||
− | | | + | |location||geometry||||||postgreSQL geometry |
− | |||
|} | |} | ||
+ | |||
+ | Note : | ||
+ | |||
+ | The parser_QC value can be 1, 2, 3, 4, 99, or 100. If it is 1 the data is OK. If it is 2 or 3 or 4, some form of error correction has been | ||
+ | performed by the Microwave parser. And if it is 100, no error correction was possible. For reliable results it is advised only to use | ||
+ | records with parser_QC values of 1. | ||
== ARGOS_TRACKING_DATA == | == ARGOS_TRACKING_DATA == | ||
Line 133: | Line 131: | ||
|'''Field'''||'''Type '''||'''Key'''||'''Constraints'''||'''Comments''' | |'''Field'''||'''Type '''||'''Key'''||'''Constraints'''||'''Comments''' | ||
|- | |- | ||
− | |'' | + | |'''ptt_id'''||int4 ||PK||NOT NULL || |
− | |||
− | |||
− | |||
− | |||
|- | |- | ||
− | |date_time ||timestamp||||NOT NULL||GMT | + | |'''date_time'''||timestamp||PK||NOT NULL||GMT |
|- | |- | ||
|fix ||text|||||| | |fix ||text|||||| | ||
Line 172: | Line 166: | ||
|'''Field'''||'''Type'''||'''Key'''||'''Constraints'''||'''Comments''' | |'''Field'''||'''Type'''||'''Key'''||'''Constraints'''||'''Comments''' | ||
|- | |- | ||
− | |'' | + | |'''ptt_id'''||int4||PK||NOT NULL || |
− | |||
− | |||
|- | |- | ||
− | | | + | |'''tx_date_time'''||timestamp||PK||NOT NULL||GMT |
|- | |- | ||
− | | | + | |'''satellite_id'''||text ||PK||NOT NULL|| |
|- | |- | ||
|ptt_date_time ||timestamp||||||GMT | |ptt_date_time ||timestamp||||||GMT | ||
− | |||
− | |||
|- | |- | ||
|activity||int4|||||| | |activity||int4|||||| | ||
Line 210: | Line 200: | ||
In addition the views GPS_TRACKING_SESSION, ARGOS_TRACKING_SESSION, PTT_ENGINEERING_SESSION are which adds the correct track_session.id to the GPS_TRACKING_DATA, ARGOS_TRACKING_DATA and PTT_ENGINEERING_DATA table data. | In addition the views GPS_TRACKING_SESSION, ARGOS_TRACKING_SESSION, PTT_ENGINEERING_SESSION are which adds the correct track_session.id to the GPS_TRACKING_DATA, ARGOS_TRACKING_DATA and PTT_ENGINEERING_DATA table data. | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
Line 312: | Line 256: | ||
|'''Field '''||'''Type'''||'''Key'''||'''Constraints'''||'''Comments''' | |'''Field '''||'''Type'''||'''Key'''||'''Constraints'''||'''Comments''' | ||
|- | |- | ||
− | |''id ''||int4||PK||NOT NULL||auto-insert, unique ID for each observation | + | |'''id '''||int4||PK||NOT NULL||auto-insert, unique ID for each observation |
|- | |- | ||
− | |'' | + | |''ring_number''||text||FK||||link to individual |
|- | |- | ||
− | | | + | |catch_datetime ||timestamp||||||Local time |
|- | |- | ||
− | + | |release_datetime||timestamp||||||Local time | |
− | |||
− | |release_datetime | ||
|- | |- | ||
|mass ||float8||||||grams | |mass ||float8||||||grams | ||
Line 338: | Line 280: | ||
|tarsus_length ||float8|||||| | |tarsus_length ||float8|||||| | ||
|- | |- | ||
− | |total_head | + | |total_head||float8|||||| |
|- | |- | ||
|observer ||text|||||| | |observer ||text|||||| | ||
|- | |- | ||
− | |remarks | + | |remarks||text|||||| |
− | |||
|} | |} | ||
Line 349: | Line 290: | ||
During ringing of juveniles only the date was recorded and not the time. Birds were all released the same day. Ringing and measuring small chicks in the nest took about 2-5 minutes. Colour ringing and sampling of the larger chicks (from 4 July onwards) took about half an hour (rounding up several chicks and banding, sampling and releasing them together). To avoid leaving fields blank, a standard catch time was noted at 12:00 and a release time of 12:15 before July 4 and 12:30 from July 4 and onwards. ''''comments by Kees Oosterbeek'''' | During ringing of juveniles only the date was recorded and not the time. Birds were all released the same day. Ringing and measuring small chicks in the nest took about 2-5 minutes. Colour ringing and sampling of the larger chicks (from 4 July onwards) took about half an hour (rounding up several chicks and banding, sampling and releasing them together). To avoid leaving fields blank, a standard catch time was noted at 12:00 and a release time of 12:15 before July 4 and 12:30 from July 4 and onwards. ''''comments by Kees Oosterbeek'''' | ||
− | |||
− | |||
== NEST_INFO == | == NEST_INFO == | ||
Line 406: | Line 345: | ||
[[Image:Gps_datamodel.png ]] | [[Image:Gps_datamodel.png ]] | ||
+ | |||
+ | |||
+ | = Argos GPS functions = | ||
+ | |||
+ | == get_argosgps_track_speed == | ||
+ | |||
+ | Compute distance (meters), time-interval, speed (meter/second) and heading (degrees) for a series of subsequent tracking entries with specific device_info_serial. | ||
+ | For the first entry in the sequence these values will always be missing because they are computed as the difference between subsequent rows. | ||
+ | |||
+ | Usage examples : | ||
+ | |||
+ | <pre> | ||
+ | select * from argos.get_argosgps_track_speed(41745, '2008-01-01 00:00:00', '2010-01-01 00:00:00'); | ||
+ | |||
+ | select * from argos.get_argosgps_track_speed(41745); | ||
+ | </pre> | ||
+ | |||
+ | Distances and heading are calculated with spherical correction with the Haversine formula as described [http://http://www.movable-type.co.uk/scripts/latlong.html here]. | ||
+ | The heading is the initial heading. Because PostGIS does not calculate the heading we don't use PostGIS here. | ||
+ | |||
+ | == get_argosgps_track_distance == | ||
+ | |||
+ | Compute distance (meters) for Argos GPS tracks to arbitrary point in lat/long or geometry location | ||
+ | |||
+ | Example usage : | ||
+ | |||
+ | <pre> | ||
+ | |||
+ | select * from argos.get_argosgps_track_distance(41745, 53.23348333,4.91656) | ||
+ | |||
+ | -- Get distances from nest | ||
+ | |||
+ | select * from argos.get_argosgps_track_distance(41745, ( | ||
+ | select n.location from argos.nest_info n, argos.track_session s, argos.individual i | ||
+ | where i.nest_info_id = n.id | ||
+ | and i.ring_number = s.ring_number | ||
+ | and s.ptt_id = 41745 limit 1 )) | ||
+ | </pre> | ||
+ | |||
+ | Here the distance is calculated with the PostGIS distance_sphere function which uses spherical correction. | ||
+ | |||
+ | == get_argosgps_track_distance_direction == | ||
+ | |||
+ | Compute distance (meters) and direction (degrees) for Argos GPS tracks from arbitrary point in lat/long or geometry location | ||
+ | |||
+ | Example usage : | ||
+ | |||
+ | <pre> | ||
+ | |||
+ | select * from argos.get_argosgps_track_distance_direction(41745, 53.23348333,4.91656) | ||
+ | |||
+ | |||
+ | -- Get distances from nest | ||
+ | create temporary table nest as | ||
+ | ( select n.latitude, n.longitude from argos.nest_info n, argos.track_session s, argos.individual i | ||
+ | where i.nest_info_id = n.id | ||
+ | and i.ring_number = s.ring_number | ||
+ | and s.ptt_id = 41745 limit 1 ); | ||
+ | |||
+ | select * from argos.get_argosgps_track_distance_direction(41745, (select latitude from nest), (select longitude from nest)); | ||
+ | |||
+ | </pre> | ||
+ | |||
+ | |||
+ | Distances and direction are calculated with spherical correction with the Haversine formula as described [http://http://www.movable-type.co.uk/scripts/latlong.html here]. The direction is the direction from the given point. |
Latest revision as of 14:11, 20 February 2018
Contents
Argos GPS Data Model
Last updated January 5, 2010 Bart Heupers - UvAGPS moved to special page
Updated January 22, 2008 Judy Shamoun-Baranes
NOTE: On July 2014 we moved all argos-related tables, views, functions and types to a new schema called argos.
e-ECOLOGY ARGOS TRACKING DATABASE
The tracking database will include data from several species. Some species are trapped and then monitored during the breeding season while others are trapped during the winter season. Information is collected on each tracked bird, including biometric measurements. These measurements may be collected more than once. During the breeding season information is also collected about the gull breeding colony itself, location of nests, state of breeding (number of eggs and juveniles) and then juveniles themselves are marked and measured (although not tracked).
In addition, PTTs are often tested before they are actually placed on the birds. This information is also important during analysis and particularly to establish accuracy of data. This data should also be stored in the database. Therefore it is important that this information is stored properly in the database
Units and descriptions should be filled in for all measurements
One way of looking at this structure is considering 3 categories of tables:
- lookup tables (information stored once)
- session tables (stored occasionally)
- observations – highest frequency of information.
Primary keys are noted by bold italic text. Foreign keys are noted by italic text in the Field column.
Microwave will under no circumstances provide information about their checksum calculations. If feasible, SARA will duplicate the procedure and include a flag representing the microwave filtered data.
TECHNICAL INFORMATION FROM TAGS
DEVICE_INFO
Table including technical information about tag hardware.
Field | Type | Key | Constraints | Comments |
ptt_id | int4 | PK | NOT NULL | serial number of PTT provided by Argos and used for communication with Argos satellite |
serial | int4 | NOT NULL | serial number of tag from manufacturer | |
weight | int4 | NOT NULL | Unit = grams | |
type | text | NOT NULL | ||
manufacturer | text | Device manufacturer | ||
hexadecimal_id | text | Hexadecimal address, unique for each tag | ||
PAN_id | Text | Personal area network ground based tags communicates with | ||
Base_station_id | Int4 | base station ground based tag communicates with | ||
Remarks | text |
TRACK_SESSION
This table includes information about each measurement session and enables inclusion of data from new species and also from the test phase of different tags
Field | Type | Key | Constraints | Comments |
id | int4 | PK | NOT NULL | unique ID for each usage of a PTT so test and bird with same ptt_id are two separate IDs/indexes |
pttid | int4 | FK | NOT NULL | Link to DEVICE_INFO table |
ring_number | text | FK | link to individual table – tests do not have a ring_number | |
start_date | timestamptz | Date/time (Local) when tracking begins | ||
start_latitude | float8 | NOT NULL | latitude where tracking begins, unit = dec degress | |
start_longitude | float8 | NOT NULL | longitude where tracking begins, unit = dec degrees | |
start_altitude | Float8 | Unit = m, include when known, important for testing | ||
end_date | timestamptz | End of session if known | ||
time_window | text | |||
time_steps | text | |||
remarks | text | |||
location | geometry | postgreSQL geometry |
GPS_TRACKING_DATA
Tracking data from GPS sensors on PTT/GPS tags
Field | Type | Key | Constraints | Comments |
ptt_id | int4 | PK | NOT NULL | |
date_time | timestamp | PK | NOT NULL | GMT |
latitude | float8 | Decimal degrees | ||
longitude | float8 | Decimal degrees | ||
speed | float8 | m/s | ||
course | float8 | degrees | ||
altitude | float8 | m | ||
alt_status | text | |||
quality | text | Quality information from sensor | ||
user_QC | text | Data flagged as unacceptable by user | ||
parser_QC | text | Data flagged as unacceptable by Microwave parser | ||
location | geometry | postgreSQL geometry |
Note :
The parser_QC value can be 1, 2, 3, 4, 99, or 100. If it is 1 the data is OK. If it is 2 or 3 or 4, some form of error correction has been performed by the Microwave parser. And if it is 100, no error correction was possible. For reliable results it is advised only to use records with parser_QC values of 1.
ARGOS_TRACKING_DATA
Tracking data from Argos sensors
Field | Type | Key | Constraints | Comments |
ptt_id | int4 | PK | NOT NULL | |
date_time | timestamp | PK | NOT NULL | GMT |
fix | text | |||
latitude1 | float8 | |||
longitude1 | float8 | |||
latitude2 | float8 | |||
longitude2 | float8 | |||
msgcount | int4 | |||
frequency | float8 | |||
average_ti | float8 | |||
satellite | text | |||
max_str | text | |||
location1 | geometry | postgreSQL geometry | ||
location2 | geometry | postgreSQL geometry |
PTT_ENGINEERING_DATA
Field | Type | Key | Constraints | Comments |
ptt_id | int4 | PK | NOT NULL | |
tx_date_time | timestamp | PK | NOT NULL | GMT |
satellite_id | text | PK | NOT NULL | |
ptt_date_time | timestamp | GMT | ||
activity | int4 | |||
tx_count | int4 | |||
temperature | float8 | |||
battery_voltage | float8 | |||
gps_fix_time | int4 | |||
satellite_count | int4 | |||
hours_reset | int4 | |||
hours_since_gps_fix | int4 | |||
mortality_gt | int4 | |||
seasonal_gt | int4 | |||
shunt | int4 |
In addition the views GPS_TRACKING_SESSION, ARGOS_TRACKING_SESSION, PTT_ENGINEERING_SESSION are which adds the correct track_session.id to the GPS_TRACKING_DATA, ARGOS_TRACKING_DATA and PTT_ENGINEERING_DATA table data.
BIOLOGICAL INFORMATION
INDIVIDUAL
Table includes information used to identify an individual
Field | Type | Key | Constraints | Comments |
ring_number | text | PK | NOT NULL | metal ring number |
Id | int4 | PK | NOT NULL | auto-insert, unique ID for each individual organism |
NEST_INFO_id | int4 | FK | link to NEST_INFO, not all specimens are linked to a nest | |
nest_certainty | int4 | 1 = certain, 0 = not completely certain | ||
species_certainty | int4 | 1 = certain, 0 = not completely certain | ||
ring_number | text | NOT NULL | metal ring number | |
color_code | text | code for color ring without punctuation | ||
species | text | NOT NULL | Scientific name (at least genus) | |
sex | text | update if or when known | ||
FATE_CODEE_id | textInt4 | FK | Link to FATE_CODE | |
Remarks | text |
FATE_code
Field | Type | Key | Constraints | Comments |
Id | Int4 | PK | auto-insert | |
code | int4 | PK | NOT NULL | |
description | text |
MEASUREMENT
Metadata Provided by Sovon is needed to describe how these measurements are taken.
Field | Type | Key | Constraints | Comments |
id | int4 | PK | NOT NULL | auto-insert, unique ID for each observation |
ring_number | text | FK | link to individual | |
catch_datetime | timestamp | Local time | ||
release_datetime | timestamp | Local time | ||
mass | float8 | grams | ||
age | text | AD = adult, JUV = juvenile | ||
wing_length | float8 | |||
bill_length1 | float8 | |||
bill_length2 | float8 | |||
bill_height1 | float8 | |||
bill_height2 | float8 | |||
tarsus_length | float8 | |||
total_head | float8 | |||
observer | text | |||
remarks | text |
Catch_datetime & release_datetime
During ringing of juveniles only the date was recorded and not the time. Birds were all released the same day. Ringing and measuring small chicks in the nest took about 2-5 minutes. Colour ringing and sampling of the larger chicks (from 4 July onwards) took about half an hour (rounding up several chicks and banding, sampling and releasing them together). To avoid leaving fields blank, a standard catch time was noted at 12:00 and a release time of 12:15 before July 4 and 12:30 from July 4 and onwards. 'comments by Kees Oosterbeek'
NEST_INFO
Field | Type | Key | Constraints | Comments |
id | int4 | PK | NOT NULL | |
Species | text | |||
latitude | float8 | NOT NULL | Decimal degrees | |
longitude | float8 | NOT NULL | Decimal degrees | |
year | Int4 | Year of breeding season | ||
"location" | geometry | postgreSQL geometry | ||
Remarks | text |
NEST_OBSERVATION
Field | Type | Key | Constraints | 'Comments' |
id | int4 | PK | NOT NULL | Auto-insert, unique ID for each observation |
NEST_INFO_id | int4 | FK | NOT NULL | |
date | timestamptz | NOT NULL | ||
number_egg | int4 | |||
number_young | int4 | |||
pipping | text | Activity in egg (peeping or signs of hatching) | ||
attended | text | Activity on nest | ||
observer | text | |||
remarks | text |
Comments:
Graphical Data Model
Argos GPS functions
get_argosgps_track_speed
Compute distance (meters), time-interval, speed (meter/second) and heading (degrees) for a series of subsequent tracking entries with specific device_info_serial. For the first entry in the sequence these values will always be missing because they are computed as the difference between subsequent rows.
Usage examples :
select * from argos.get_argosgps_track_speed(41745, '2008-01-01 00:00:00', '2010-01-01 00:00:00'); select * from argos.get_argosgps_track_speed(41745);
Distances and heading are calculated with spherical correction with the Haversine formula as described here. The heading is the initial heading. Because PostGIS does not calculate the heading we don't use PostGIS here.
get_argosgps_track_distance
Compute distance (meters) for Argos GPS tracks to arbitrary point in lat/long or geometry location
Example usage :
select * from argos.get_argosgps_track_distance(41745, 53.23348333,4.91656) -- Get distances from nest select * from argos.get_argosgps_track_distance(41745, ( select n.location from argos.nest_info n, argos.track_session s, argos.individual i where i.nest_info_id = n.id and i.ring_number = s.ring_number and s.ptt_id = 41745 limit 1 ))
Here the distance is calculated with the PostGIS distance_sphere function which uses spherical correction.
get_argosgps_track_distance_direction
Compute distance (meters) and direction (degrees) for Argos GPS tracks from arbitrary point in lat/long or geometry location
Example usage :
select * from argos.get_argosgps_track_distance_direction(41745, 53.23348333,4.91656) -- Get distances from nest create temporary table nest as ( select n.latitude, n.longitude from argos.nest_info n, argos.track_session s, argos.individual i where i.nest_info_id = n.id and i.ring_number = s.ring_number and s.ptt_id = 41745 limit 1 ); select * from argos.get_argosgps_track_distance_direction(41745, (select latitude from nest), (select longitude from nest));
Distances and direction are calculated with spherical correction with the Haversine formula as described here. The direction is the direction from the given point.