Difference between revisions of "Argos Data Model"

From ecology
Jump to: navigation, search
(get_argosgps_track_distance_direction)
(GPS Data Model)
 
(10 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== GPS Data Model ==  
+
== Argos GPS Data Model ==  
  
 
Last updated January 5, 2010 Bart Heupers - [[ UvAGPS ]] moved to special page
 
Last updated January 5, 2010 Bart Heupers - [[ UvAGPS ]] moved to special page
Line 5: Line 5:
 
Updated January 22, 2008 Judy Shamoun-Baranes  
 
Updated January 22, 2008 Judy Shamoun-Baranes  
  
'''FLYSAFE TRACKING DATABASE'''
+
'''NOTE:''' On July 2014 we moved all argos-related tables, views, functions and types to a new schema called '''argos'''.
 +
 
 +
'''e-ECOLOGY ARGOS TRACKING DATABASE'''
  
 
The tracking database will include data from several species. Some species are trapped and then monitored during the breeding season while others are trapped during the winter season. Information is collected on each tracked bird, including biometric measurements. These measurements may be collected more than once. During the breeding season information is also collected about the gull breeding colony itself, location of nests, state of breeding (number of eggs and juveniles) and then juveniles themselves are marked and measured (although not tracked).
 
The tracking database will include data from several species. Some species are trapped and then monitored during the breeding season while others are trapped during the winter season. Information is collected on each tracked bird, including biometric measurements. These measurements may be collected more than once. During the breeding season information is also collected about the gull breeding colony itself, location of nests, state of breeding (number of eggs and juveniles) and then juveniles themselves are marked and measured (although not tracked).
Line 21: Line 23:
 
Primary keys are noted by bold italic text. Foreign keys are noted by italic text in the Field column.
 
Primary keys are noted by bold italic text. Foreign keys are noted by italic text in the Field column.
  
Microwave, will under no circumstances provide information about their checksum calculations. If feasible, SARA will duplicate the procedure and include a flag in the representing the microwave filtered data.
+
Microwave will under no circumstances provide information about their checksum calculations. If feasible, SARA will duplicate the procedure and include a flag representing the microwave filtered data.
 
 
 
'''TECHNICAL INFORMATION FROM TAGS'''
 
'''TECHNICAL INFORMATION FROM TAGS'''
 
  
 
== DEVICE_INFO ==
 
== DEVICE_INFO ==
Line 119: Line 120:
 
Note :  
 
Note :  
  
The parser_QC value can be 1, 2,3,4, 99, or 100. If it 1 the data is OK. If it is 2 or 3 or 4 some form of error correction has been  
+
The parser_QC value can be 1, 2, 3, 4, 99, or 100. If it is 1 the data is OK. If it is 2 or 3 or 4, some form of error correction has been  
performed by the Microwave parser. And if it is 100 no error correction was possible. For reliable results it is advised only to use  
+
performed by the Microwave parser. And if it is 100, no error correction was possible. For reliable results it is advised only to use  
records with parser_QC values of 1
+
records with parser_QC values of 1.
  
 
== ARGOS_TRACKING_DATA ==
 
== ARGOS_TRACKING_DATA ==
Line 356: Line 357:
  
 
<pre>
 
<pre>
select * from gps.get_argosgps_track_speed(41745, '2008-01-01 00:00:00', '2010-01-01 00:00:00');
+
select * from argos.get_argosgps_track_speed(41745, '2008-01-01 00:00:00', '2010-01-01 00:00:00');
  
select * from gps.get_argosgps_track_speed(41745);
+
select * from argos.get_argosgps_track_speed(41745);
 
</pre>
 
</pre>
  
Line 372: Line 373:
 
<pre>
 
<pre>
  
select * from gps.get_argosgps_track_distance(41745, 53.23348333,4.91656)
+
select * from argos.get_argosgps_track_distance(41745, 53.23348333,4.91656)
  
 
-- Get distances from nest
 
-- Get distances from nest
  
select * from gps.get_argosgps_track_distance(41745, (
+
select * from argos.get_argosgps_track_distance(41745, (
   select n.location from gps.nest_info n, gps.track_session s, gps.individual i  
+
   select n.location from argos.nest_info n, argos.track_session s, argos.individual i  
 
     where i.nest_info_id = n.id  
 
     where i.nest_info_id = n.id  
 
       and i.ring_number = s.ring_number  
 
       and i.ring_number = s.ring_number  
Line 393: Line 394:
 
<pre>
 
<pre>
  
select * from gps.get_argosgps_track_distance_direction(41745, 53.23348333,4.91656)
+
select * from argos.get_argosgps_track_distance_direction(41745, 53.23348333,4.91656)
  
  
 
-- Get distances from nest
 
-- Get distances from nest
 
create temporary table nest as
 
create temporary table nest as
(  select  n.latitude, n.longitude from gps.nest_info n, gps.track_session s, gps.individual i  
+
(  select  n.latitude, n.longitude from argos.nest_info n, argos.track_session s, argos.individual i  
 
     where i.nest_info_id = n.id  
 
     where i.nest_info_id = n.id  
 
       and i.ring_number = s.ring_number  
 
       and i.ring_number = s.ring_number  
 
       and s.ptt_id = 41745 limit 1 );
 
       and s.ptt_id = 41745 limit 1 );
 
        
 
        
select * from gps.get_argosgps_track_distance_direction(41745, (select latitude from nest), (select longitude from nest));
+
select * from argos.get_argosgps_track_distance_direction(41745, (select latitude from nest), (select longitude from nest));
  
 
</pre>
 
</pre>

Latest revision as of 14:11, 20 February 2018

Argos GPS Data Model

Last updated January 5, 2010 Bart Heupers - UvAGPS moved to special page

Updated January 22, 2008 Judy Shamoun-Baranes

NOTE: On July 2014 we moved all argos-related tables, views, functions and types to a new schema called argos.

e-ECOLOGY ARGOS TRACKING DATABASE

The tracking database will include data from several species. Some species are trapped and then monitored during the breeding season while others are trapped during the winter season. Information is collected on each tracked bird, including biometric measurements. These measurements may be collected more than once. During the breeding season information is also collected about the gull breeding colony itself, location of nests, state of breeding (number of eggs and juveniles) and then juveniles themselves are marked and measured (although not tracked).

In addition, PTTs are often tested before they are actually placed on the birds. This information is also important during analysis and particularly to establish accuracy of data. This data should also be stored in the database. Therefore it is important that this information is stored properly in the database

Units and descriptions should be filled in for all measurements

One way of looking at this structure is considering 3 categories of tables:

  1. lookup tables (information stored once)
  2. session tables (stored occasionally)
  3. observations – highest frequency of information.

Primary keys are noted by bold italic text. Foreign keys are noted by italic text in the Field column.

Microwave will under no circumstances provide information about their checksum calculations. If feasible, SARA will duplicate the procedure and include a flag representing the microwave filtered data.

TECHNICAL INFORMATION FROM TAGS

DEVICE_INFO

Table including technical information about tag hardware.

Field Type Key Constraints Comments
ptt_id int4 PK NOT NULL serial number of PTT provided by Argos and used for communication with Argos satellite
serial int4 NOT NULL serial number of tag from manufacturer
weight int4 NOT NULL Unit = grams
type text NOT NULL
manufacturer text Device manufacturer
hexadecimal_id text Hexadecimal address, unique for each tag
PAN_id Text Personal area network ground based tags communicates with
Base_station_id Int4 base station ground based tag communicates with
Remarks text

TRACK_SESSION

This table includes information about each measurement session and enables inclusion of data from new species and also from the test phase of different tags

Field Type Key Constraints Comments
id int4 PK NOT NULL unique ID for each usage of a PTT so test and bird with same ptt_id are two separate IDs/indexes
pttid int4 FK NOT NULL Link to DEVICE_INFO table
ring_number text FK link to individual table – tests do not have a ring_number
start_date timestamptz Date/time (Local) when tracking begins
start_latitude float8 NOT NULL latitude where tracking begins, unit = dec degress
start_longitude float8 NOT NULL longitude where tracking begins, unit = dec degrees
start_altitude Float8 Unit = m, include when known, important for testing
end_date timestamptz End of session if known
time_window text
time_steps text
remarks text
location geometry postgreSQL geometry

GPS_TRACKING_DATA

Tracking data from GPS sensors on PTT/GPS tags

Field Type Key Constraints Comments
ptt_id int4 PK NOT NULL
date_time timestamp PK NOT NULL GMT
latitude float8 Decimal degrees
longitude float8 Decimal degrees
speed float8 m/s
course float8 degrees
altitude float8 m
alt_status text
quality text Quality information from sensor
user_QC text Data flagged as unacceptable by user
parser_QC text Data flagged as unacceptable by Microwave parser
location geometry postgreSQL geometry


Note :

The parser_QC value can be 1, 2, 3, 4, 99, or 100. If it is 1 the data is OK. If it is 2 or 3 or 4, some form of error correction has been performed by the Microwave parser. And if it is 100, no error correction was possible. For reliable results it is advised only to use records with parser_QC values of 1.

ARGOS_TRACKING_DATA

Tracking data from Argos sensors

Field Type Key Constraints Comments
ptt_id int4 PK NOT NULL
date_time timestamp PK NOT NULL GMT
fix text
latitude1 float8
longitude1 float8
latitude2 float8
longitude2 float8
msgcount int4
frequency float8
average_ti float8
satellite text
max_str text
location1 geometry postgreSQL geometry
location2 geometry postgreSQL geometry

PTT_ENGINEERING_DATA

Field Type Key Constraints Comments
ptt_id int4 PK NOT NULL
tx_date_time timestamp PK NOT NULL GMT
satellite_id text PK NOT NULL
ptt_date_time timestamp GMT
activity int4
tx_count int4
temperature float8
battery_voltage float8
gps_fix_time int4
satellite_count int4
hours_reset int4
hours_since_gps_fix int4
mortality_gt int4
seasonal_gt int4
shunt int4


In addition the views GPS_TRACKING_SESSION, ARGOS_TRACKING_SESSION, PTT_ENGINEERING_SESSION are which adds the correct track_session.id to the GPS_TRACKING_DATA, ARGOS_TRACKING_DATA and PTT_ENGINEERING_DATA table data.


BIOLOGICAL INFORMATION

INDIVIDUAL

Table includes information used to identify an individual

Field Type Key Constraints Comments
ring_number text PK NOT NULL metal ring number
Id int4 PK NOT NULL auto-insert, unique ID for each individual organism
NEST_INFO_id int4 FK link to NEST_INFO, not all specimens are linked to a nest
nest_certainty int4 1 = certain, 0 = not completely certain
species_certainty int4 1 = certain, 0 = not completely certain
ring_number text NOT NULL metal ring number
color_code text code for color ring without punctuation
species text NOT NULL Scientific name (at least genus)
sex text update if or when known
FATE_CODEE_id textInt4 FK Link to FATE_CODE
Remarks text

FATE_code

Field Type Key Constraints Comments
Id Int4 PK auto-insert
code int4 PK NOT NULL
description text


MEASUREMENT

Metadata Provided by Sovon is needed to describe how these measurements are taken.

Field Type Key Constraints Comments
id int4 PK NOT NULL auto-insert, unique ID for each observation
ring_number text FK link to individual
catch_datetime timestamp Local time
release_datetime timestamp Local time
mass float8 grams
age text AD = adult, JUV = juvenile
wing_length float8
bill_length1 float8
bill_length2 float8
bill_height1 float8
bill_height2 float8
tarsus_length float8
total_head float8
observer text
remarks text

Catch_datetime & release_datetime

During ringing of juveniles only the date was recorded and not the time. Birds were all released the same day. Ringing and measuring small chicks in the nest took about 2-5 minutes. Colour ringing and sampling of the larger chicks (from 4 July onwards) took about half an hour (rounding up several chicks and banding, sampling and releasing them together). To avoid leaving fields blank, a standard catch time was noted at 12:00 and a release time of 12:15 before July 4 and 12:30 from July 4 and onwards. 'comments by Kees Oosterbeek'

NEST_INFO

Field Type Key Constraints Comments
id int4 PK NOT NULL
Species text
latitude float8 NOT NULL Decimal degrees
longitude float8 NOT NULL Decimal degrees
year Int4 Year of breeding season
"location" geometry postgreSQL geometry
Remarks text

NEST_OBSERVATION

Field Type Key Constraints 'Comments'
id int4 PK NOT NULL Auto-insert, unique ID for each observation
NEST_INFO_id int4 FK NOT NULL
date timestamptz NOT NULL
number_egg int4
number_young int4
pipping text Activity in egg (peeping or signs of hatching)
attended text Activity on nest
observer text
remarks text

Comments:

Graphical Data Model

Gps datamodel.png


Argos GPS functions

get_argosgps_track_speed

Compute distance (meters), time-interval, speed (meter/second) and heading (degrees) for a series of subsequent tracking entries with specific device_info_serial. For the first entry in the sequence these values will always be missing because they are computed as the difference between subsequent rows.

Usage examples :

select * from argos.get_argosgps_track_speed(41745, '2008-01-01 00:00:00', '2010-01-01 00:00:00');

select * from argos.get_argosgps_track_speed(41745);

Distances and heading are calculated with spherical correction with the Haversine formula as described here. The heading is the initial heading. Because PostGIS does not calculate the heading we don't use PostGIS here.

get_argosgps_track_distance

Compute distance (meters) for Argos GPS tracks to arbitrary point in lat/long or geometry location

Example usage :


select * from argos.get_argosgps_track_distance(41745, 53.23348333,4.91656)

-- Get distances from nest

select * from argos.get_argosgps_track_distance(41745, (
  select n.location from argos.nest_info n, argos.track_session s, argos.individual i 
    where i.nest_info_id = n.id 
      and i.ring_number = s.ring_number 
      and s.ptt_id = 41745 limit 1 ))

Here the distance is calculated with the PostGIS distance_sphere function which uses spherical correction.

get_argosgps_track_distance_direction

Compute distance (meters) and direction (degrees) for Argos GPS tracks from arbitrary point in lat/long or geometry location

Example usage :


select * from argos.get_argosgps_track_distance_direction(41745, 53.23348333,4.91656)


-- Get distances from nest
create temporary table nest as
(   select  n.latitude, n.longitude from argos.nest_info n, argos.track_session s, argos.individual i 
    where i.nest_info_id = n.id 
      and i.ring_number = s.ring_number 
      and s.ptt_id = 41745 limit 1 );
      
select * from argos.get_argosgps_track_distance_direction(41745, (select latitude from nest), (select longitude from nest));


Distances and direction are calculated with spherical correction with the Haversine formula as described here. The direction is the direction from the given point.