https://wiki.e-ecology.nl/api.php?action=feedcontributions&feedformat=atom&user=Stefan+verhoevenecology - User contributions [en]2026-04-06T11:37:53ZUser contributionsMediaWiki 1.30.0https://wiki.e-ecology.nl/index.php?title=Useful_Queries&diff=1271Useful Queries2015-10-13T09:29:40Z<p>Stefan verhoeven: /* Re-sample tracking data to select data with a predefined interval between fixes */ Do resample only on requested trackers</p>
<hr />
<div>= UvAGPS functions =<br />
<br />
== get_uvagps_track_speed ==<br />
<br />
Compute distance (meters), time-interval (time stamp hh:mm:ss), speed (meters/second) and heading (degrees) for a series of subsequent tracking entries for a specific device_info_serial. <br />
For the first entry in the sequence these values will always be missing because they are computed as the difference between subsequent rows.<br />
<br />
Usage examples :<br />
<br />
<pre><br />
select * from gps.get_uvagps_track_speed(<device_info_serial>);<br />
<br />
select * from gps.get_uvagps_track_speed(119, '2008-01-01 00:00:00', '2010-01-01 00:00:00');<br />
<br />
select t.*, a.distance, a.interval, a.speed, a.direction <br />
from gps.get_uvagps_track_speed(119, '2008-01-01 00:00:00', '2010-01-01 00:00:00') a <br />
join gps.ee_tracking_speed_limited t using(device_info_serial, date_time);<br />
</pre><br />
<br />
Distances and heading are calculated with spherical correction with the Haversine formula as described [http://http://www.movable-type.co.uk/scripts/latlong.html here]. <br />
The heading is the initial heading. PostGIS does not calculate the heading, so therefore we don't use PostGIS.<br />
<br />
In these functions the records that have been flagged as invalid by setting the '''userflag''' to 1 are not taken into account and discarded. <br />
If speed calculations are required for all records an additional boolean argument '''get raw data''' can be given. i.e.<br />
<br />
<pre><br />
select * from gps.get_uvagps_track_speed(119, '2008-01-01 00:00:00', '2010-01-01 00:00:00', true);<br />
<br />
select * from gps.get_uvagps_track_speed(119, true);<br />
</pre><br />
<br />
== get_uvagps_track_distance ==<br />
<br />
Compute distance (meters) for UvAGPS tracks to arbitrary point in lat/long or geometry location <br />
<br />
Example usage :<br />
<br />
<pre><br />
select * from gps.get_uvagps_track_distance(119, 52.1897017, 6.1996606); <br />
<br />
select * from gps.get_uvagps_track_distance(119, (select location from gps.ee_tracking_speed_limited where device_info_serial = 119 and date_time = '2009-05-07 19:55:08')); <br />
<br />
</pre><br />
<br />
Here the distance is calculated with the PostGIS distance_sphere function which uses spherical correction.<br />
<br />
== get_uvagps_track_distance_direction ==<br />
<br />
<br />
Compute distance (meters) and direction (degrees) for UvAGPS tracks from arbitrary point in lat/long or geometry location<br />
<br />
Example usage: <br />
<br />
<pre><br />
select * from gps.get_uvagps_track_distance_direction(119, 52.1897017, 6.1996606); <br />
</pre><br />
<br />
Distances and direction are calculated with spherical correction with the Haversine formula as described [http://http://www.movable-type.co.uk/scripts/latlong.html here]. The direction is the direction from the given point.<br />
<br />
In this function the records where the '''userflag''' has been set are discarded. To see these records a additional boolean argument for '''show raw data''' should be added like :<br />
<br />
<pre><br />
select * from gps.get_uvagps_track_distance_direction(119, 52.1897017, 6.1996606, true); <br />
</pre><br />
= Filtering tracking data =<br />
<br />
== Select tracking data from the beginning of a track session ==<br />
<br />
Select only those points that were collected during a valid track session (i.e. data that begins from the start_date of a track session) including 2D GPS speed. Filter our points where no GPS fix is returned (latitude is not null) and where a user has flagged the data as inappropriate (user_flag <> 1):<br />
<br />
SELECT t.device_info_serial, t.date_time, t.latitude, t.longitude, t.altitude, t.temperature, t.speed, t.userflag<br />
FROM gps.ee_tracking_speed_limited t, gps.ee_track_session_limited s<br />
WHERE t.device_info_serial = s.device_info_serial AND t.date_time >= s.start_date AND t.device_info_serial = '''<ID>'''<br />
AND t.latitude IS NOT NULL AND t.userflag <> 1<br />
<br />
== Select tracking data and distinguish between distinct GPS deployments ==<br />
In this query you will select the tracking data related to each deployment of a GPS logger or each time an individual bird was tracked. This query is slightly more complex than the query above. If you have used the same logger on more than one individual bird, or have tracked the same individual with more than one logger, this query will enable you to make a distinction between these tracking events by showing both GPS logger id (device_info_serial) and a unique identifier for your individual (eg ring_number or colour_ring). Furthermore, as above, this will only select tracking data from the start and end of a particular track session.<br />
<br />
SELECT t.device_info_serial, t.date_time, t.latitude, t.longitude, t.altitude, t.speed_2d, i.colour_ring, s.track_session_id, s.key_name<br />
FROM gps.ee_track_session_limited s<br />
JOIN gps.ee_individual_limited i<br />
ON s.individual_id = i.individual_id<br />
JOIN gps.ee_tracking_speed_limited t<br />
ON t.device_info_serial = s.device_info_serial<br />
AND t.date_time BETWEEN s.start_date AND s.end_date<br />
WHERE s.key_name = ''''<Project name>'''' AND t.userflag = 0<br />
ORDER BY t.device_info_serial, t.date_time<br />
<br />
HINT: WHERE s.key_name = 'HG_TEXEL'<br />
<br />
== Re-sample tracking data to select data with a predefined interval between fixes ==<br />
<br />
The following query provides an example of how to select data points with a predefined interval between each point and not lower. In the following example, a 5 minute interval is selected. In this example all measurements that are taken between each 5 minute interval are left out of the resampled data. <br />
<br />
Commented query with examples. Statements followed by -- are comments and are not parsed by SQL.<br />
<br />
SELECT *<br />
FROM gps.ee_tracking_speed_limited<br />
JOIN<br />
(<br />
SELECT <br />
device_info_serial, <br />
-- return oldest date_time for each interval<br />
MIN(date_time) date_time,<br />
-- each 5 minutes has a unique identifier<br />
FLOOR(EXTRACT(EPOCH FROM date_time)/(5*60)) intervalid<br />
FROM gps.ee_tracking_speed_limited<br />
WHERE device_info_serial = 355 AND date_time BETWEEN '2010-06-27' AND '2010-06-29'<br />
GROUP BY device_info_serial, intervalid<br />
) b<br />
USING (device_info_serial, date_time)<br />
<br />
ORDER BY date_time<br />
<br />
Uncommented query, date_time and device_info_serial need to be filled in. Interval is 5 minutes and can be changed.<br />
<br />
SELECT *<br />
FROM gps.ee_tracking_speed_limited<br />
JOIN<br />
(<br />
SELECT <br />
device_info_serial, <br />
MIN(date_time) date_time,<br />
FLOOR(EXTRACT(EPOCH FROM date_time)/(5*60)) intervalid<br />
FROM gps.ee_tracking_speed_limited<br />
WHERE device_info_serial = '''<ID>''' AND date_time BETWEEN ''''yyyy-mm-dd'''' AND ''''yyyy-mm-dd''''<br />
GROUP BY device_info_serial, intervalid<br />
) b<br />
USING (device_info_serial, date_time)<br />
ORDER BY date_time<br />
<br />
= Calculating summary statistics =<br />
== Total distance ==<br />
A query to determine the total distance a bird has flown is : <br />
<br />
select sum(distance)/1000 as total_distance<br />
from gps.get_uvagps_track_speed('''<ID>''');<br />
<br />
or for a fixed time period :<br />
<br />
select sum(distance)/1000 as total_distance<br />
from gps.get_uvagps_track_speed('''<ID>''', '2010-01-01 00:00:00', '2011-01-01 00:00:00');<br />
<br />
== Total number of measurements per tag ==<br />
<br />
Count the total number of measurements for each device using the GROUP BY clause:<br />
<br />
select device_info_serial, count(*) <br />
from gps.ee_tracking_speed_limited<br />
group by device_info_serial<br />
ORDER BY device_info_serial ASC; <br />
<br />
<br />
If you want to count the number of measurements with valid GPS fixes you can add a where clause :<br />
<br />
select device_info_serial, count(*) <br />
from gps.ee_tracking_speed_limited<br />
where latitude is not null <br />
group by device_info_serial<br />
ORDER BY device_info_serial ASC;<br />
<br />
Count the number of GPS fixes within a track session and for all track sessions within a specific project.<br />
In the example below the project name needs to be filled in, for example s.key_name = 'HG_IVES'<br />
<br />
WITH <br />
V AS (<br />
SELECT s.device_info_serial, count(t.*) AS point_count, s.track_session_id, s.individual_id<br />
FROM gps.ee_track_session_limited s<br />
JOIN gps.ee_tracking_speed_limited t<br />
ON t.latitude IS NOT NULL<br />
AND s.device_info_serial = t.device_info_serial <br />
AND t.date_time BETWEEN s.start_date AND s.end_date<br />
WHERE s.key_name = ''''< project name >''''<br />
GROUP BY s.track_session_id, s.device_info_serial, s.individual_id<br />
)<br />
SELECT i.colour_ring, V.device_info_serial, V.point_count, V.track_session_id<br />
FROM V<br />
JOIN gps.ee_individual_limited i<br />
ON V.individual_id = i.individual_id<br />
= Extra Metadata =<br />
== Extract all track sessions for a project ==<br />
This query provides all the track sessions for a specific project as well as the selected ID of the individual birds that were tracked. In the example below the project name needs to be filled in, for example s.key_name = 'HG_IVES'<br />
<br />
SELECT s.device_info_serial, i.colour_ring, s.track_session_id, s.key_name<br />
FROM gps.ee_individual_limited i<br />
JOIN gps.ee_track_session_limited s<br />
ON s.individual_id = i.individual_id<br />
WHERE s.key_name = ''''< project name >'''' <br />
ORDER BY i.colour_ring</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=CartoDB&diff=1266CartoDB2015-09-11T13:59:16Z<p>Stefan verhoeven: Update url</p>
<hr />
<div>CartoDB is an open source tool that allows for the storage and visualization of geospatial data on the web. See http://www.cartodb.com<br />
<br />
Our own CartoDB server is at http://cartodb.e-ecology.vm.surfsara.nl .<br />
Accounts can be made on request by s.verhoeven@esciencecenter.nl after being OK-ed by Willem Bouten.<br />
<br />
Installation documentation of CartoDB server can be found at https://github.com/NLeSC/eEcology-CartoDB<br />
<br />
== GPS tables ==<br />
<br />
The following tables are available on the CartoDB server:<br />
<br />
* gps.ee_species_limited<br />
* gps.ee_individual_limited<br />
* gps.ee_project_limited<br />
* gps.ee_tracker_limited<br />
* gps.ee_track_session_limited<br />
* gps.ee_tracking_speed_limited<br />
* gps.ee_acceleration_limited<br />
* gps.ee_acc_start_limited<br />
* gps.ee_nest_limited<br />
* gps.ee_nest_inhabitant_limited<br />
<br />
The tables have the same colums as described on [[DB_Views_2015]]<br />
<br />
Tables with a location column have been made CartoDB compliant by<br />
<br />
* Adding cartodb_id column, as primary key<br />
* Adding the_geom_webmercator column<br />
* Renaming the location column to the_geom column<br />
<br />
Only data from the following projects is available on the CartoDB server:<br />
<br />
* lbbg_texel <br />
* hg_texel<br />
* hb_veluwe<br />
* ef_alegranza<br />
<br />
The data is not synced.<br />
<br />
== Limitations ==<br />
<br />
* Disk space is limited, please do not copy data from gps.* tables to your own account<br />
* Some links will redirect back to http://cartodb.com<br />
* Whenever you see an error or have a support question please don't contact support at cartodb, but contact s.verhoeven@esciencecenter.nl<br />
* Geocoding (eg. converting city name to lat,lon) is not working<br />
* Login with Google+ does not work<br />
* Making visualization of a lot of rows (more than 65000 rows per tile) can be very slow<br />
* Export image does not work. Make screenshot of web page as a workaround<br />
<br />
== Create CartoDB map visualization based on eEcology data ==<br />
<br />
# Press `NEW MAP` button<br />
# Press `CREATE NEW MAP` button<br />
# Press `CREATE EMPTY MAP` button<br />
# Enter a SQL query in the SQL tab (SQL icon left side of screen) <br />
# Press `Apply query` button<br />
# Switch to `MAP VIEW` to see the map<br />
<br />
=== Example queries ===<br />
<br />
A time window of a single tracker<br />
<br />
<pre><br />
SELECT * <br />
FROM gps.ee_tracking_speed_limited <br />
WHERE<br />
device_info_serial = 344<br />
AND<br />
date_time BETWEEN '2013-01-01' AND '2014-01-01'<br />
</pre><br />
<br />
Compare 2 species <br />
<br />
<pre><br />
SELECT cartodb_id, the_geom, the_geom_webmercator, date_time, english_name<br />
FROM<br />
gps.ee_species_limited s<br />
JOIN gps.ee_individual_limited i ON s.latin_name = i.species_latin_name<br />
JOIN gps.ee_track_session_limited USING (ring_number)<br />
JOIN gps.ee_tracking_speed_limited USING (device_info_serial)<br />
WHERE<br />
date_time BETWEEN '2015-01-01' AND '2015-06-02'<br />
AND<br />
english_name IN ('Herring Gull', 'Lesser Black-backed Gull')<br />
</pre><br />
<br />
== Share visualizations and datasets ==<br />
<br />
Sharing has several different levels:<br />
<br />
* public, visualization is listed on your public profile page<br />
* public if url is known<br />
* readable by logged in users<br />
* private</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=CartoDB&diff=1265CartoDB2015-08-18T09:45:51Z<p>Stefan verhoeven: Added export image kaput</p>
<hr />
<div>CartoDB is an open source tool that allows for the storage and visualization of geospatial data on the web. See http://www.cartodb.com<br />
<br />
Our own CartoDB server is at http://cartodb.e-ecology.cloudlet.sara.nl .<br />
Accounts can be made on request by s.verhoeven@esciencecenter.nl after being OK-ed by Willem Bouten.<br />
<br />
Installation documentation of CartoDB server can be found at https://github.com/NLeSC/eEcology-CartoDB<br />
<br />
== GPS tables ==<br />
<br />
The following tables are available on the CartoDB server:<br />
<br />
* gps.ee_species_limited<br />
* gps.ee_individual_limited<br />
* gps.ee_project_limited<br />
* gps.ee_tracker_limited<br />
* gps.ee_track_session_limited<br />
* gps.ee_tracking_speed_limited<br />
* gps.ee_acceleration_limited<br />
* gps.ee_acc_start_limited<br />
* gps.ee_nest_limited<br />
* gps.ee_nest_inhabitant_limited<br />
<br />
The tables have the same colums as described on [[DB_Views_2015]]<br />
<br />
Tables with a location column have been made CartoDB compliant by<br />
<br />
* Adding cartodb_id column, as primary key<br />
* Adding the_geom_webmercator column<br />
* Renaming the location column to the_geom column<br />
<br />
Only data from the following projects is available on the CartoDB server:<br />
<br />
* lbbg_texel <br />
* hg_texel<br />
* hb_veluwe<br />
* ef_alegranza<br />
<br />
The data is not synced.<br />
<br />
== Limitations ==<br />
<br />
* Disk space is limited, please do not copy data from gps.* tables to your own account<br />
* Some links will redirect back to http://cartodb.com<br />
* Whenever you see an error or have a support question please don't contact support at cartodb, but contact s.verhoeven@esciencecenter.nl<br />
* Geocoding (eg. converting city name to lat,lon) is not working<br />
* Login with Google+ does not work<br />
* Making visualization of a lot of rows (more than 65000 rows per tile) can be very slow<br />
* Export image does not work. Make screenshot of web page as a workaround<br />
<br />
== Create CartoDB map visualization based on eEcology data ==<br />
<br />
# Press `NEW MAP` button<br />
# Press `CREATE NEW MAP` button<br />
# Press `CREATE EMPTY MAP` button<br />
# Enter a SQL query in the SQL tab (SQL icon left side of screen) <br />
# Press `Apply query` button<br />
# Switch to `MAP VIEW` to see the map<br />
<br />
=== Example queries ===<br />
<br />
A time window of a single tracker<br />
<br />
<pre><br />
SELECT * <br />
FROM gps.ee_tracking_speed_limited <br />
WHERE<br />
device_info_serial = 344<br />
AND<br />
date_time BETWEEN '2013-01-01' AND '2014-01-01'<br />
</pre><br />
<br />
Compare 2 species <br />
<br />
<pre><br />
SELECT cartodb_id, the_geom, the_geom_webmercator, date_time, english_name<br />
FROM<br />
gps.ee_species_limited s<br />
JOIN gps.ee_individual_limited i ON s.latin_name = i.species_latin_name<br />
JOIN gps.ee_track_session_limited USING (ring_number)<br />
JOIN gps.ee_tracking_speed_limited USING (device_info_serial)<br />
WHERE<br />
date_time BETWEEN '2015-01-01' AND '2015-06-02'<br />
AND<br />
english_name IN ('Herring Gull', 'Lesser Black-backed Gull')<br />
</pre><br />
<br />
== Share visualizations and datasets ==<br />
<br />
Sharing has several different levels:<br />
<br />
* public, visualization is listed on your public profile page<br />
* public if url is known<br />
* readable by logged in users<br />
* private</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=CartoDB&diff=1264CartoDB2015-08-04T08:46:39Z<p>Stefan verhoeven: </p>
<hr />
<div>CartoDB is an open source tool that allows for the storage and visualization of geospatial data on the web. See http://www.cartodb.com<br />
<br />
Our own CartoDB server is at http://cartodb.e-ecology.cloudlet.sara.nl .<br />
Accounts can be made on request by s.verhoeven@esciencecenter.nl after being OK-ed by Willem Bouten.<br />
<br />
Installation documentation of CartoDB server can be found at https://github.com/NLeSC/eEcology-CartoDB<br />
<br />
== GPS tables ==<br />
<br />
The following tables are available on the CartoDB server:<br />
<br />
* gps.ee_species_limited<br />
* gps.ee_individual_limited<br />
* gps.ee_project_limited<br />
* gps.ee_tracker_limited<br />
* gps.ee_track_session_limited<br />
* gps.ee_tracking_speed_limited<br />
* gps.ee_acceleration_limited<br />
* gps.ee_acc_start_limited<br />
* gps.ee_nest_limited<br />
* gps.ee_nest_inhabitant_limited<br />
<br />
The tables have the same colums as described on [[DB_Views_2015]]<br />
<br />
Tables with a location column have been made CartoDB compliant by<br />
<br />
* Adding cartodb_id column, as primary key<br />
* Adding the_geom_webmercator column<br />
* Renaming the location column to the_geom column<br />
<br />
Only data from the following projects is available on the CartoDB server:<br />
<br />
* lbbg_texel <br />
* hg_texel<br />
* hb_veluwe<br />
* ef_alegranza<br />
<br />
The data is not synced.<br />
<br />
== Limitations ==<br />
<br />
* Disk space is limited, please do not copy data from gps.* tables to your own account<br />
* Some links will redirect back to http://cartodb.com<br />
* Whenever you see an error or have a support question please don't contact support at cartodb, but contact s.verhoeven@esciencecenter.nl<br />
* Geocoding (eg. converting city name to lat,lon) is not working<br />
* Login with Google+ does not work<br />
* Making visualization of a lot of rows (more than 65000 rows per tile) can be very slow<br />
<br />
== Create CartoDB map visualization based on eEcology data ==<br />
<br />
# Press `NEW MAP` button<br />
# Press `CREATE NEW MAP` button<br />
# Press `CREATE EMPTY MAP` button<br />
# Enter a SQL query in the SQL tab (SQL icon left side of screen) <br />
# Press `Apply query` button<br />
# Switch to `MAP VIEW` to see the map<br />
<br />
=== Example queries ===<br />
<br />
A time window of a single tracker<br />
<br />
<pre><br />
SELECT * <br />
FROM gps.ee_tracking_speed_limited <br />
WHERE<br />
device_info_serial = 344<br />
AND<br />
date_time BETWEEN '2013-01-01' AND '2014-01-01'<br />
</pre><br />
<br />
Compare 2 species <br />
<br />
<pre><br />
SELECT cartodb_id, the_geom, the_geom_webmercator, date_time, english_name<br />
FROM<br />
gps.ee_species_limited s<br />
JOIN gps.ee_individual_limited i ON s.latin_name = i.species_latin_name<br />
JOIN gps.ee_track_session_limited USING (ring_number)<br />
JOIN gps.ee_tracking_speed_limited USING (device_info_serial)<br />
WHERE<br />
date_time BETWEEN '2015-01-01' AND '2015-06-02'<br />
AND<br />
english_name IN ('Herring Gull', 'Lesser Black-backed Gull')<br />
</pre><br />
<br />
== Share visualizations and datasets ==<br />
<br />
Sharing has several different levels:<br />
<br />
* public, visualization is listed on your public profile page<br />
* public if url is known<br />
* readable by logged in users<br />
* private</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=CartoDB&diff=1263CartoDB2015-08-04T07:45:21Z<p>Stefan verhoeven: /* GPS tables */ More tables copied</p>
<hr />
<div>CartoDB is an open source tool that allows for the storage and visualization of geospatial data on the web. See http://www.cartodb.com<br />
<br />
Our own CartoDB server is at http://cartodb.e-ecology.cloudlet.sara.nl .<br />
Accounts can be made on request by s.verhoeven@esciencecenter.nl after being OK-ed by Willem Bouten.<br />
<br />
Documentation for setup of CartoDB server can be found at https://github.com/NLeSC/eEcology-CartoDB<br />
<br />
== GPS tables ==<br />
<br />
The following tables are available on the CartoDB server:<br />
<br />
* gps.ee_species_limited<br />
* gps.ee_individual_limited<br />
* gps.ee_project_limited<br />
* gps.ee_tracker_limited<br />
* gps.ee_track_session_limited<br />
* gps.ee_tracking_speed_limited<br />
* gps.ee_acceleration_limited<br />
* gps.ee_acc_start_limited<br />
* gps.ee_nest_limited<br />
* gps.ee_nest_inhabitant_limited<br />
<br />
The tables have the same colums as described on [[DB_Views_2015]]<br />
<br />
Tables with a location column have been made CartoDB compliant by<br />
<br />
* Adding cartodb_id column, as primary key<br />
* Adding the_geom_webmercator column<br />
* Renaming the location column to the_geom column<br />
<br />
Only data from the following projects is available on the CartoDB server:<br />
<br />
* lbbg_texel <br />
* hg_texel<br />
* hb_veluwe<br />
* ef_alegranza<br />
<br />
The data is not synced.<br />
<br />
== Limitations ==<br />
<br />
* Disk space is limited, please do not copy data from gps.* tables to your own account<br />
* Some links will redirect back to http://cartodb.com<br />
* Whenever you see an error or have a support question please don't contact support at cartodb, but contact s.verhoeven@esciencecenter.nl<br />
* Geocoding (eg. converting city name to lat,lon) is not working<br />
* Login with Google+ does not work<br />
<br />
== Create CartoDB map visualization based on eEcology data ==<br />
<br />
# Press `NEW MAP` button<br />
# Press `CREATE NEW MAP` button<br />
# Press `CREATE EMPTY MAP` button<br />
# Enter a SQL query in the SQL tab (SQL icon left side of screen) <br />
# Press `Apply query` button<br />
# Switch to `MAP VIEW` to see the map<br />
<br />
=== Example queries ===<br />
<br />
A time window of a single tracker<br />
<br />
<pre><br />
SELECT * <br />
FROM gps.ee_tracking_speed_limited <br />
WHERE<br />
device_info_serial = 344<br />
AND<br />
date_time BETWEEN '2013-01-01' AND '2014-01-01'<br />
</pre><br />
<br />
Compare 2 species <br />
<br />
<pre><br />
SELECT cartodb_id, the_geom, the_geom_webmercator, date_time, english_name<br />
FROM<br />
gps.ee_species_limited s<br />
JOIN gps.ee_individual_limited i ON s.latin_name = i.species_latin_name<br />
JOIN gps.ee_track_session_limited USING (ring_number)<br />
JOIN gps.ee_tracking_speed_limited USING (device_info_serial)<br />
WHERE<br />
date_time BETWEEN '2015-01-01' AND '2015-06-02'<br />
AND<br />
english_name IN ('Herring Gull', 'Lesser Black-backed Gull')<br />
</pre><br />
<br />
== Share visualizations and datasets ==<br />
<br />
Sharing has several different levels:<br />
<br />
* public<br />
* public if url is known<br />
* readable by CartoDB users<br />
* private</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=CartoDB&diff=1262CartoDB2015-07-30T13:42:10Z<p>Stefan verhoeven: /* Example queries */</p>
<hr />
<div>CartoDB is an open source tool that allows for the storage and visualization of geospatial data on the web. See http://www.cartodb.com<br />
<br />
Our own CartoDB server is at http://cartodb.e-ecology.cloudlet.sara.nl .<br />
Accounts can be made on request by s.verhoeven@esciencecenter.nl after being OK-ed by Willem Bouten.<br />
<br />
Documentation for setup of CartoDB server can be found at https://github.com/NLeSC/eEcology-CartoDB<br />
<br />
== GPS tables ==<br />
<br />
The following tables are available on the CartoDB server:<br />
<br />
* gps.ee_species_limited<br />
* gps.ee_individual_limited<br />
* gps.ee_project_limited<br />
* gps.ee_tracker_limited<br />
* gps.ee_track_session_limited<br />
* gps.ee_tracking_speed_limited<br />
<br />
The tables have the same colums as described on [[DB_Views_2015]]<br />
<br />
Tables with a location column have been made CartoDB compliant by<br />
<br />
* Adding cartodb_id column, as primary key<br />
* Adding the_geom_webmercator column<br />
* Renaming the location column to the_geom column<br />
<br />
Only data from the following projects is available on the CartoDB server:<br />
<br />
* lbbg_texel <br />
* hg_texel<br />
* hb_veluwe<br />
* ef_alegranza<br />
<br />
The data is not synced.<br />
<br />
== Limitations ==<br />
<br />
* Disk space is limited, please do not copy data from gps.* tables to your own account<br />
* Some links will redirect back to http://cartodb.com<br />
* Whenever you see an error or have a support question please don't contact support at cartodb, but contact s.verhoeven@esciencecenter.nl<br />
* Geocoding (eg. converting city name to lat,lon) is not working<br />
* Login with Google+ does not work<br />
<br />
== Create CartoDB map visualization based on eEcology data ==<br />
<br />
# Press `NEW MAP` button<br />
# Press `CREATE NEW MAP` button<br />
# Press `CREATE EMPTY MAP` button<br />
# Enter a SQL query in the SQL tab (SQL icon left side of screen) <br />
# Press `Apply query` button<br />
# Switch to `MAP VIEW` to see the map<br />
<br />
=== Example queries ===<br />
<br />
A time window of a single tracker<br />
<br />
<pre><br />
SELECT * <br />
FROM gps.ee_tracking_speed_limited <br />
WHERE<br />
device_info_serial = 344<br />
AND<br />
date_time BETWEEN '2013-01-01' AND '2014-01-01'<br />
</pre><br />
<br />
Compare 2 species <br />
<br />
<pre><br />
SELECT cartodb_id, the_geom, the_geom_webmercator, date_time, english_name<br />
FROM<br />
gps.ee_species_limited s<br />
JOIN gps.ee_individual_limited i ON s.latin_name = i.species_latin_name<br />
JOIN gps.ee_track_session_limited USING (ring_number)<br />
JOIN gps.ee_tracking_speed_limited USING (device_info_serial)<br />
WHERE<br />
date_time BETWEEN '2015-01-01' AND '2015-06-02'<br />
AND<br />
english_name IN ('Herring Gull', 'Lesser Black-backed Gull')<br />
</pre><br />
<br />
== Share visualizations and datasets ==<br />
<br />
Sharing has several different levels:<br />
<br />
* public<br />
* public if url is known<br />
* readable by CartoDB users<br />
* private</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=CartoDB&diff=1261CartoDB2015-07-30T13:33:14Z<p>Stefan verhoeven: /* Example queries */</p>
<hr />
<div>CartoDB is an open source tool that allows for the storage and visualization of geospatial data on the web. See http://www.cartodb.com<br />
<br />
Our own CartoDB server is at http://cartodb.e-ecology.cloudlet.sara.nl .<br />
Accounts can be made on request by s.verhoeven@esciencecenter.nl after being OK-ed by Willem Bouten.<br />
<br />
Documentation for setup of CartoDB server can be found at https://github.com/NLeSC/eEcology-CartoDB<br />
<br />
== GPS tables ==<br />
<br />
The following tables are available on the CartoDB server:<br />
<br />
* gps.ee_species_limited<br />
* gps.ee_individual_limited<br />
* gps.ee_project_limited<br />
* gps.ee_tracker_limited<br />
* gps.ee_track_session_limited<br />
* gps.ee_tracking_speed_limited<br />
<br />
The tables have the same colums as described on [[DB_Views_2015]]<br />
<br />
Tables with a location column have been made CartoDB compliant by<br />
<br />
* Adding cartodb_id column, as primary key<br />
* Adding the_geom_webmercator column<br />
* Renaming the location column to the_geom column<br />
<br />
Only data from the following projects is available on the CartoDB server:<br />
<br />
* lbbg_texel <br />
* hg_texel<br />
* hb_veluwe<br />
* ef_alegranza<br />
<br />
The data is not synced.<br />
<br />
== Limitations ==<br />
<br />
* Disk space is limited, please do not copy data from gps.* tables to your own account<br />
* Some links will redirect back to http://cartodb.com<br />
* Whenever you see an error or have a support question please don't contact support at cartodb, but contact s.verhoeven@esciencecenter.nl<br />
* Geocoding (eg. converting city name to lat,lon) is not working<br />
* Login with Google+ does not work<br />
<br />
== Create CartoDB map visualization based on eEcology data ==<br />
<br />
# Press `NEW MAP` button<br />
# Press `CREATE NEW MAP` button<br />
# Press `CREATE EMPTY MAP` button<br />
# Enter a SQL query in the SQL tab (SQL icon left side of screen) <br />
# Press `Apply query` button<br />
# Switch to `MAP VIEW` to see the map<br />
<br />
=== Example queries ===<br />
<br />
A time window of a single tracker<br />
<br />
<code><br />
SELECT * <br />
FROM gps.ee_tracking_speed_limited <br />
WHERE<br />
device_info_serial = 344<br />
AND<br />
date_time BETWEEN '2013-01-01' AND '2014-01-01'<br />
</code><br />
<br />
Compare 2 species <br />
<br />
<code><br />
SELECT cartodb_id, the_geom, the_geom_webmercator, date_time, english_name<br />
FROM<br />
gps.ee_species_limited s<br />
JOIN gps.ee_individual_limited i ON s.latin_name = i.species_latin_name<br />
JOIN gps.ee_track_session_limited USING (ring_number)<br />
JOIN gps.ee_tracking_speed_limited USING (device_info_serial)<br />
WHERE<br />
date_time BETWEEN '2015-01-01' AND '2015-06-02'<br />
AND<br />
english_name IN ('Herring Gull', 'Lesser Black-backed Gull')<br />
</code></div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=CartoDB&diff=1260CartoDB2015-07-29T15:35:02Z<p>Stefan verhoeven: /* Create CartoDB map visualization based on eEcology data */</p>
<hr />
<div>CartoDB is an open source tool that allows for the storage and visualization of geospatial data on the web. See http://www.cartodb.com<br />
<br />
Our own CartoDB server is at http://cartodb.e-ecology.cloudlet.sara.nl .<br />
Accounts can be made on request by s.verhoeven@esciencecenter.nl after being OK-ed by Willem Bouten.<br />
<br />
Documentation for setup of CartoDB server can be found at https://github.com/NLeSC/eEcology-CartoDB<br />
<br />
== GPS tables ==<br />
<br />
The following tables are available on the CartoDB server:<br />
<br />
* gps.ee_species_limited<br />
* gps.ee_individual_limited<br />
* gps.ee_project_limited<br />
* gps.ee_tracker_limited<br />
* gps.ee_track_session_limited<br />
* gps.ee_tracking_speed_limited<br />
<br />
The tables have the same colums as described on [[DB_Views_2015]]<br />
<br />
Tables with a location column have been made CartoDB compliant by<br />
<br />
* Adding cartodb_id column, as primary key<br />
* Adding the_geom_webmercator column<br />
* Renaming the location column to the_geom column<br />
<br />
Only data from the following projects is available on the CartoDB server:<br />
<br />
* lbbg_texel <br />
* hg_texel<br />
* hb_veluwe<br />
* ef_alegranza<br />
<br />
The data is not synced.<br />
<br />
== Limitations ==<br />
<br />
* Disk space is limited, please do not copy data from gps.* tables to your own account<br />
* Some links will redirect back to http://cartodb.com<br />
* Whenever you see an error or have a support question please don't contact support at cartodb, but contact s.verhoeven@esciencecenter.nl<br />
* Geocoding (eg. converting city name to lat,lon) is not working<br />
* Login with Google+ does not work<br />
<br />
== Create CartoDB map visualization based on eEcology data ==<br />
<br />
# Press `NEW MAP` button<br />
# Press `CREATE NEW MAP` button<br />
# Press `CREATE EMPTY MAP` button<br />
# Enter a SQL query in the SQL tab (SQL icon left side of screen) <br />
# Press `Apply query` button<br />
# Switch to `MAP VIEW` to see the map<br />
<br />
=== Example queries ===<br />
<br />
A time window of a single tracker<br />
<code><br />
SELECT * <br />
FROM gps.ee_tracking_speed_limited <br />
WHERE<br />
device_info_serial = 355<br />
AND<br />
date_time BETWEEN '2011-05-26' AND '2011-05-26'<br />
</code></div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=CartoDB&diff=1259CartoDB2015-07-29T14:44:12Z<p>Stefan verhoeven: </p>
<hr />
<div>CartoDB is an open source tool that allows for the storage and visualization of geospatial data on the web. See http://www.cartodb.com<br />
<br />
Our own CartoDB server is at http://cartodb.e-ecology.cloudlet.sara.nl .<br />
Accounts can be made on request by s.verhoeven@esciencecenter.nl after being OK-ed by Willem Bouten.<br />
<br />
Documentation for setup of CartoDB server can be found at https://github.com/NLeSC/eEcology-CartoDB<br />
<br />
== GPS tables ==<br />
<br />
The following tables are available on the CartoDB server:<br />
<br />
* gps.ee_species_limited<br />
* gps.ee_individual_limited<br />
* gps.ee_project_limited<br />
* gps.ee_tracker_limited<br />
* gps.ee_track_session_limited<br />
* gps.ee_tracking_speed_limited<br />
<br />
The tables have the same colums as described on [[DB_Views_2015]]<br />
<br />
Tables with a location column have been made CartoDB compliant by<br />
<br />
* Adding cartodb_id column, as primary key<br />
* Adding the_geom_webmercator column<br />
* Renaming the location column to the_geom column<br />
<br />
Only data from the following projects is available on the CartoDB server:<br />
<br />
* lbbg_texel <br />
* hg_texel<br />
* hb_veluwe<br />
* ef_alegranza<br />
<br />
The data is not synced.<br />
<br />
== Limitations ==<br />
<br />
* Disk space is limited, please do not copy data from gps.* tables to your own account<br />
* Some links will redirect back to http://cartodb.com<br />
* Whenever you see an error or have a support question please don't contact support at cartodb, but contact s.verhoeven@esciencecenter.nl<br />
* Geocoding (eg. converting city name to lat,lon) is not working<br />
* Login with Google+ does not work<br />
<br />
== Create CartoDB map visualization based on eEcology data ==<br />
<br />
1. Press `NEW MAP` button<br />
2. Press `CREATE NEW MAP` button<br />
3. Press `CREATE EMPTY MAP` button<br />
4. Enter a SQL query in the SQL tab (SQL icon left side of screen) <br />
5. Press `Apply query` button<br />
6. Switch to `MAP VIEW` to see the map<br />
<br />
Do not press `create dataset from query` link as this will duplicate data.<br />
<br />
=== Example queries ===<br />
<br />
A time window of a single tracker<br />
<code><br />
SELECT * <br />
FROM gps.ee_tracking_speed_limited <br />
WHERE<br />
device_info_serial = 355<br />
AND<br />
date_time BETWEEN '2011-05-26' AND '2011-05-26'<br />
</code></div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=CartoDB&diff=1258CartoDB2015-07-28T15:47:00Z<p>Stefan verhoeven: </p>
<hr />
<div>CartoDB is an open source tool that allows for the storage and visualization of geospatial data on the web. See http://www.cartodb.com<br />
<br />
Our own CartoDB server is at http://cartodb.e-ecology.cloudlet.sara.nl .<br />
Accounts can be made on request by s.verhoeven@esciencecenter.nl after being OK-ed by Willem Bouten.<br />
<br />
Documentation for setup of CartoDB server can be found at https://github.com/NLeSC/eEcology-CartoDB<br />
<br />
== GPS tables ==<br />
<br />
The following tables are available on the CartoDB server:<br />
<br />
* gps.ee_species_limited<br />
* gps.ee_individual_limited<br />
* gps.ee_project_limited<br />
* gps.ee_tracker_limited<br />
* gps.ee_track_session_limited<br />
* gps.ee_tracking_speed_limited<br />
<br />
The tables have the same colums as described on [[DB_Views_2015]]<br />
<br />
Tables with a location column have been made CartoDB compliant by<br />
<br />
* Adding cartodb_id column, as primary key<br />
* Adding the_geom_webmercator column<br />
* Renaming the location column to the_geom column<br />
<br />
Only data from the following projects is available on the CartoDB server:<br />
<br />
* lbbg_texel <br />
* hg_texel<br />
* hb_veluwe<br />
* ef_alegranza<br />
<br />
The data is not synced.<br />
<br />
== Limitations ==<br />
<br />
* Some links will redirect back to http://cartodb.com<br />
* Whenever you see an error or have a support question please don't contact support at cartodb, but contact s.verhoeven@esciencecenter.nl<br />
* Geocoding (eg. converting city name to lat,lon) is not working<br />
* Login with Google+ does not work</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=CartoDB&diff=1257CartoDB2015-07-28T15:01:01Z<p>Stefan verhoeven: /* Limitations */</p>
<hr />
<div>CartoDB is an open source tool that allows for the storage and visualization of geospatial data on the web. See http://www.cartodb.com<br />
<br />
Our own CartoDB server is at http://cartodb.e-ecology.cloudlet.sara.nl .<br />
Accounts can be made on request by s.verhoeven@esciencecenter.nl after being OK-ed by Willem Bouten.<br />
<br />
Documentation for setup of CartoDB server can be found at https://github.com/NLeSC/eEcology-CartoDB<br />
<br />
== GPS tables ==<br />
<br />
The following tables are available on the CartoDB server:<br />
<br />
* gps.ee_species_limited<br />
* gps.ee_individual_limited<br />
* gps.ee_project_limited<br />
* gps.ee_tracker_limited<br />
* gps.ee_track_session_limited<br />
* gps.ee_tracking_speed_limited<br />
<br />
The tables have the same colums as described on https://services.e-ecology.sara.nl/wiki/index.php/DB_Views_2015<br />
<br />
Tables with a location column have been made CartoDB compliant by<br />
<br />
* Adding cartodb_id column, as primary key<br />
* Adding the_geom_webmercator column<br />
* Renaming the location column to the_geom column<br />
<br />
Only data from the following projects is available on the CartoDB server:<br />
<br />
* lbbg_texel <br />
* hg_texel<br />
* hb_veluwe<br />
* ef_alegranza<br />
<br />
The data is not synced.<br />
<br />
== Limitations ==<br />
<br />
* Some links will redirect back to http://cartodb.com<br />
* Whenever you see an error or have a support question please don't contact support at cartodb, but contact s.verhoeven@esciencecenter.nl<br />
* Geocoding (eg. converting city name to lat,lon) is not working<br />
* Login with Google+ does not work</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=CartoDB&diff=1256CartoDB2015-07-28T14:55:32Z<p>Stefan verhoeven: /* Limitations */</p>
<hr />
<div>CartoDB is an open source tool that allows for the storage and visualization of geospatial data on the web. See http://www.cartodb.com<br />
<br />
Our own CartoDB server is at http://cartodb.e-ecology.cloudlet.sara.nl .<br />
Accounts can be made on request by s.verhoeven@esciencecenter.nl after being OK-ed by Willem Bouten.<br />
<br />
Documentation for setup of CartoDB server can be found at https://github.com/NLeSC/eEcology-CartoDB<br />
<br />
== GPS tables ==<br />
<br />
The following tables are available on the CartoDB server:<br />
<br />
* gps.ee_species_limited<br />
* gps.ee_individual_limited<br />
* gps.ee_project_limited<br />
* gps.ee_tracker_limited<br />
* gps.ee_track_session_limited<br />
* gps.ee_tracking_speed_limited<br />
<br />
The tables have the same colums as described on https://services.e-ecology.sara.nl/wiki/index.php/DB_Views_2015<br />
<br />
Tables with a location column have been made CartoDB compliant by<br />
<br />
* Adding cartodb_id column, as primary key<br />
* Adding the_geom_webmercator column<br />
* Renaming the location column to the_geom column<br />
<br />
Only data from the following projects is available on the CartoDB server:<br />
<br />
* lbbg_texel <br />
* hg_texel<br />
* hb_veluwe<br />
* ef_alegranza<br />
<br />
The data is not synced.<br />
<br />
== Limitations ==<br />
<br />
* Uploading a dataset is not working<br />
* Some links will redirect back to http://cartodb.com<br />
* Whenever you see an error or have a support question please don't contact support at cartodb, but contact s.verhoeven@esciencecenter.nl<br />
* Geocoding (eg. converting city name to lat,lon) is not working<br />
* Login with Google+ does not work</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=CartoDB&diff=1255CartoDB2015-07-28T14:52:10Z<p>Stefan verhoeven: </p>
<hr />
<div>CartoDB is an open source tool that allows for the storage and visualization of geospatial data on the web. See http://www.cartodb.com<br />
<br />
Our own CartoDB server is at http://cartodb.e-ecology.cloudlet.sara.nl .<br />
Accounts can be made on request by s.verhoeven@esciencecenter.nl after being OK-ed by Willem Bouten.<br />
<br />
Documentation for setup of CartoDB server can be found at https://github.com/NLeSC/eEcology-CartoDB<br />
<br />
== GPS tables ==<br />
<br />
The following tables are available on the CartoDB server:<br />
<br />
* gps.ee_species_limited<br />
* gps.ee_individual_limited<br />
* gps.ee_project_limited<br />
* gps.ee_tracker_limited<br />
* gps.ee_track_session_limited<br />
* gps.ee_tracking_speed_limited<br />
<br />
The tables have the same colums as described on https://services.e-ecology.sara.nl/wiki/index.php/DB_Views_2015<br />
<br />
Tables with a location column have been made CartoDB compliant by<br />
<br />
* Adding cartodb_id column, as primary key<br />
* Adding the_geom_webmercator column<br />
* Renaming the location column to the_geom column<br />
<br />
Only data from the following projects is available on the CartoDB server:<br />
<br />
* lbbg_texel <br />
* hg_texel<br />
* hb_veluwe<br />
* ef_alegranza<br />
<br />
The data is not synced.<br />
<br />
== Limitations ==<br />
<br />
* Some links will redirect back to http://cartodb.com<br />
* Whenever you see an error or have a support question please don't contact support at cartodb, but contact s.verhoeven@esciencecenter.nl<br />
* Geocoding (eg. converting city name to lat,lon) is not working<br />
* Login with Google+ does not work</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=CartoDB&diff=1254CartoDB2015-07-27T15:54:30Z<p>Stefan verhoeven: </p>
<hr />
<div>CartoDB is an open source tool that allows for the storage and visualization of geospatial data on the web. See http://www.cartodb.com<br />
<br />
Our own CartoDB server is at http://cartodb.e-ecology.cloudlet.sara.nl .<br />
Accounts can be made on request by s.verhoeven@esciencecenter.nl after being OK-ed by Willem Bouten.<br />
<br />
Documentation for setup of CartoDB server can be found at https://github.com/NLeSC/eEcology-CartoDB<br />
<br />
== GPS tables ==<br />
<br />
The e-ecology gps.ee*limited tables are available on the CartoDB server.<br />
The tables available are the same as on https://services.e-ecology.sara.nl/wiki/index.php/DB_Views_2015, but with following changes to tables with a location column:<br />
<br />
* Added cartodb_id column, as primary key<br />
* Added the_geom_webmercator column<br />
* Renamed location column to the_geom column<br />
<br />
Only data from the gull projects is available on the CartoDB server.<br />
The data is not synced.<br />
<br />
== Limitations ==<br />
<br />
* Not all links are working like:<br />
<br />
* Whenever you see an error please don't contact Cartodb.com, but contact s.verhoeven@esciencecenter.nl</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=CartoDB&diff=1253CartoDB2015-07-27T10:45:18Z<p>Stefan verhoeven: Created page with "CartoDB is an open source tool that allows for the storage and visualization of geospatial data on the web. See http://www.cartodb.com Our own CartoDB server is at http://cartod..."</p>
<hr />
<div>CartoDB is an open source tool that allows for the storage and visualization of geospatial data on the web. See http://www.cartodb.com<br />
<br />
Our own CartoDB server is at http://cartodb.e-ecology.cloudlet.sara.nl .<br />
Accounts can be made on request by s.verhoeven@esciencecenter.nl after being OK-ed by Willem Bouten.<br />
<br />
== GPS tables ==<br />
<br />
The e-ecology gps.ee*limited tables are available on the CartoDB server.<br />
The tables available are the same as on https://services.e-ecology.sara.nl/wiki/index.php/DB_Views_2015, but with following changes to tables with a location column:<br />
<br />
* Added cartodb_id column, as primary key<br />
* Added the_geom_webmercator column<br />
* Renamed location column to the_geom column<br />
<br />
Only data from the gull projects is available on the CartoDB server.<br />
The data is not synced.</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=E-Ecology_Documentation_and_Guidance&diff=1252E-Ecology Documentation and Guidance2015-07-27T10:38:04Z<p>Stefan verhoeven: </p>
<hr />
<div>The e-Ecology Virtual Laboratories represent a collaborative effort between the University of Amsterdam Institute for Biodiversity and Ecosystem Dynamics and SARA Computing to manage bird tracking data and develop tools for data exploration, analysis and visualization. This wiki is designed to facilitate access to the data and tools within the Virtual Laboratories and to provide a forum for collaboration. <br />
<br />
== UvA-BiTS data == <br />
<br />
:Bird data collected with the UvA-BiTS Bird Tracking System<br />
<br />
Note: We are continually working on improving UvA-BiTS services, including data processing and Virtual Lab functionality. In some cases, this has enabled us to extract more information from old GPS log files which is then incorporated into the database. These changes may result in slight differences when you reprocess historic data. <br />
<br />
::* [[ DB_Views_2015 | e-Ecology database for general use (views) ]]<br />
::* [[ DB_Tables_2015 | e-Ecology tables behind the views (not accessible for normal use) ]]<br />
::* [[ UvAGPS|UvA-BiTS Data Model]] - OLD<br />
::* [[Connecting to the UvA-BiTS database]]<br />
::* [[Useful Queries]]<br />
::* [[Queries for linking bird tracks to wind and flow data]]<br />
<br />
==UvA-BiTS Services==<br />
<br />
* Visualization Tools<br />
:: [[BirdView]]: <br /><br />
:::Access meterological and radar data and overlay with GPS bird tracking data.<br />
:: [[BirdSim]]: <br/ ><br />
:::More dynamic version of BirdView that allows the user to scroll through weather data in time and space.<br />
:: [[KMZ Generator|UvA-BiTS KMZ Generator]]: <br /><br />
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth. <br />
:: [[Tracker Calendar]]: <br /><br />
:::Calendar overview with daily statistics of GPS-tracker<br />
:: [[Flight Generator]]: <br /><br />
:::This tool produces GPX files from UvA-BiTS tracking data for dynamic [http://www.doarama.com Doarama] visualizations.<br />
<br />
* [[Dashboard]]: These web pages allow UvA-BiTS users to get an overview of the raw data files of GPS bird tracking devices they have access to and to access the dashboard for administering projects. <br />
<br />
* [[ Annotation tables ]] for uploading GPS annotation data into your own schema. This service is currently not available.<br />
* [[ SMS tables ]] for storage of SMS messages send by trackers.<br />
* [[ CartoDB ]] usage and limitation of CartoDB server<br />
<br />
== Changes in the Database in 2014 ==<br />
'''Important'''<br />
In the summer of 2014 we have '''re-arranged the administrative''' part of UvA-BiTS (and e-Ecology) in the database. End users may have to adapt their programs the following way:<br />
:* The '''uva_*_limited''' views that used to exist in the gps schema are gone. Most of them should have been replaced with an equivalent '''ee_*_limited counterpart'''. <br />
:* The new ee_*_limited views show a superset of the same information as before, but enhanced with new project information.<br />
:* Some '''column names''' in the ee_*_limited views may differ from the previous views.<br />
:* There are new limited views: <br />
:** gps.ee_'''project'''_limited: shows your projects' information<br />
:** gps.ee_'''shared'''_*_limited: set of data you are able to see related to trackers that a project is sharing with a project you belong to<br />
:* Special attention: gps.uva_device_limited is now replaced by '''gps.ee_tracker_limited'''. For each tracker, you now see which project provides you access to that tracker.<br />
:* Also, '''gps.uva_tracking_limited''' is gone. Instead of that we have gps.ee_tracker_speed_limited, which adds speed columns to the original data. This amounts to a merger of gps.ee_tracking_limited, gps.ee_tracking_speed_3d_limited and gps.ee_tracking_speed_limited.<br />
<br />
==Other e-Ecology Datasets==<br />
<br />
'''ENVIRONMENTAL DATA'''<br />
<br />
::* [[ ECMWF|Weather Forecast Data (ECMWF)]]<br />
::* [[ Elevation|Elevation Data ]]<br />
<br />
'''FLYSAFE'''<br />
<br />
:Bird data created from weather radar data and bird migration models. Financed by the Royal Netherlands Air Force. <br />
<br />
::* [[Connecting to the Flysafe database]]<br />
::* [[GPS|Flysafe GPS Data Model]]<br />
::* [[ RADAR|Radar Data Model]]<br />
::* [[ Wind profile|Wind Profile Data Model]]<br />
::* [[ Fino1|Fino1 Data Model]]<br />
<br />
==Other e-Ecology Services==<br />
[[GPS tracks|FlySafe GPS Bird Track Generator]]: Produce kmz files of ARGOS/GPS bird tracking data from the FlySafe project to be viewed in Google Earth. <br />
<br />
[[ Administration]]: Information about user rights and data processing tasks.<br />
<br />
[[ Bookkeeping]]: Description of tables related to FlySafe data processing and archiving.<br />
<br />
[[ ToDo ]]</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=SMS_tables&diff=1251SMS tables2015-05-11T11:28:45Z<p>Stefan verhoeven: </p>
<hr />
<div>The eEcology SMS reciever (https://github.com/NLeSC/eEcology-SMS-reciever) service recieves SMS messages as http POST requests and stores them in the eEcology database.<br />
The SMS messages are converted to http POST requests by the SMSSync Android app (http://smssync.ushahidi.com/).<br />
<br />
The messages are stored in the PostgreSQL server at db.e-ecology.sara.nl in the eecology database and '''sms''' schema.<br />
The data can be inserted by the eEcology SMS reciever service.<br />
The data can be selected by users with the gps_read role.<br />
<br />
Data is stored in 3 tables:<br />
<br />
= sms.raw_message table =<br />
<br />
Data received from sending party. <br />
<br />
<table xmlns="http://www.w3.org/1999/xhtml" border=1><br />
<tr><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Name</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Data type</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Not Null?</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Primary key?</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Default</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Comment</th><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">id</td><br />
<td class="ReportTableValueCell">integer</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell">Auto incrementing</td><br />
<td class="ReportTableValueCell"> </td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">message_id</td><br />
<td class="ReportTableValueCell">uuid</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">unique ID of the SMS</td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">sent_from</td><br />
<td class="ReportTableValueCell">character varying</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">number that sent the SMS</td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">body</td><br />
<td class="ReportTableValueCell">character varying</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Body of the SMS</td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">sent_to</td><br />
<td class="ReportTableValueCell">character varying</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">the phone number registered on the SIM card otherwise it's the value set on the app as device ID</td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">gateway_id</td><br />
<td class="ReportTableValueCell">character varying</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">the unique id set on the device to be used by the server to identify which device is communicating with it </td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">sent_timestamp</td><br />
<td class="ReportTableValueCell">timestamp with time zone</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">the timestamp the SMS was sent</td><br />
</tr><br />
</table><br />
<br />
= sms.message table =<br />
<br />
The parsed message body.<br />
<br />
<table xmlns="http://www.w3.org/1999/xhtml" border=1><br />
<tr><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Name</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Data type</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Not Null?</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Primary key?</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Default</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Comment</th><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">id</td><br />
<td class="ReportTableValueCell">integer</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Foreign key sms.raw_message.id</td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">device_info_serial</td><br />
<td class="ReportTableValueCell">integer</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Tracker identifier</td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">date_time</td><br />
<td class="ReportTableValueCell">timestamp with time zone</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Datetime the message was recieved</td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">battery_voltage</td><br />
<td class="ReportTableValueCell">real</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell"> </td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">memory_usage</td><br />
<td class="ReportTableValueCell">real</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell"> </td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">debug_info</td><br />
<td class="ReportTableValueCell">character varying</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell"> </td><br />
</tr><br />
</table><br />
<br />
= sms.position table =<br />
<br />
The positions parsed form the message body.<br />
<br />
<table xmlns="http://www.w3.org/1999/xhtml" border=1><br />
<tr><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Name</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Data type</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Not Null?</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Primary key?</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Default</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Comment</th><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">id</td><br />
<td class="ReportTableValueCell">integer</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Foreign key sms.message.id </td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">device_info_serial</td><br />
<td class="ReportTableValueCell">integer</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Tracker identifier</td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">date_time</td><br />
<td class="ReportTableValueCell">timestamp with time zone</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Timestamp at which location was taken</td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">lon</td><br />
<td class="ReportTableValueCell">double precision</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Longitude </td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">lat</td><br />
<td class="ReportTableValueCell">double precision</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Latitude</td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">location</td><br />
<td class="ReportTableValueCell">geometry(1107456)</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">PostGIS location as a Point(lon, lat) </td><br />
</tr><br />
</table></div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=SMS_tables&diff=1250SMS tables2015-05-11T11:21:10Z<p>Stefan verhoeven: </p>
<hr />
<div>The eEcology SMS reciever (https://github.com/NLeSC/eEcology-SMS-reciever) service recieves SMS messages as http POST requests and stores them in the eEcology database.<br />
<br />
The messages are stored in the PostgreSQL server at db.e-ecology.sara.nl in the eecology database and '''sms''' schema.<br />
The data can be inserted by the eEcology SMS reciever service.<br />
The data can be selected by users with the gps_read role.<br />
<br />
Data is stored in 3 tables:<br />
<br />
= sms.raw_message table =<br />
<br />
Data received from sending party. <br />
<br />
<table xmlns="http://www.w3.org/1999/xhtml" border=1><br />
<tr><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Name</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Data type</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Not Null?</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Primary key?</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Default</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Comment</th><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">id</td><br />
<td class="ReportTableValueCell">integer</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell">Auto incrementing</td><br />
<td class="ReportTableValueCell"> </td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">message_id</td><br />
<td class="ReportTableValueCell">uuid</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">unique ID of the SMS</td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">sent_from</td><br />
<td class="ReportTableValueCell">character varying</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">number that sent the SMS</td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">body</td><br />
<td class="ReportTableValueCell">character varying</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Body of the SMS</td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">sent_to</td><br />
<td class="ReportTableValueCell">character varying</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">the phone number registered on the SIM card otherwise it's the value set on the app as device ID</td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">gateway_id</td><br />
<td class="ReportTableValueCell">character varying</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">the unique id set on the device to be used by the server to identify which device is communicating with it </td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">sent_timestamp</td><br />
<td class="ReportTableValueCell">timestamp with time zone</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">the timestamp the SMS was sent</td><br />
</tr><br />
</table><br />
<br />
= sms.message table =<br />
<br />
The parsed message body.<br />
<br />
<table xmlns="http://www.w3.org/1999/xhtml" border=1><br />
<tr><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Name</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Data type</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Not Null?</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Primary key?</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Default</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Comment</th><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">id</td><br />
<td class="ReportTableValueCell">integer</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Foreign key sms.raw_message.id</td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">device_info_serial</td><br />
<td class="ReportTableValueCell">integer</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Tracker identifier</td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">date_time</td><br />
<td class="ReportTableValueCell">timestamp with time zone</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Datetime the message was recieved</td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">battery_voltage</td><br />
<td class="ReportTableValueCell">real</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell"> </td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">memory_usage</td><br />
<td class="ReportTableValueCell">real</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell"> </td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">debug_info</td><br />
<td class="ReportTableValueCell">character varying</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell"> </td><br />
</tr><br />
</table><br />
<br />
= sms.position table =<br />
<br />
The positions parsed form the message body.<br />
<br />
<table xmlns="http://www.w3.org/1999/xhtml" border=1><br />
<tr><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Name</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Data type</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Not Null?</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Primary key?</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Default</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Comment</th><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">id</td><br />
<td class="ReportTableValueCell">integer</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Foreign key sms.message.id </td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">device_info_serial</td><br />
<td class="ReportTableValueCell">integer</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Tracker identifier</td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">date_time</td><br />
<td class="ReportTableValueCell">timestamp with time zone</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Timestamp at which location was taken</td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">lon</td><br />
<td class="ReportTableValueCell">double precision</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Longitude </td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">lat</td><br />
<td class="ReportTableValueCell">double precision</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Latitude</td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">location</td><br />
<td class="ReportTableValueCell">geometry(1107456)</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">PostGIS location as a Point(lon, lat) </td><br />
</tr><br />
</table></div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=SMS_tables&diff=1249SMS tables2015-05-11T11:19:21Z<p>Stefan verhoeven: </p>
<hr />
<div>The eEcology SMS reciever (https://github.com/NLeSC/eEcology-SMS-reciever) service recieves SMS messages as http POST requests and stores them in the eEcology database.<br />
<br />
The messages are stored in the PostgreSQL server at db.e-ecology.sara.nl in the eecology database and '''sms''' schema.<br />
The data can be inserted by the eEcology SMS reciever service.<br />
The data can be selected by users with the gps_read role.<br />
<br />
Data is stored in 3 tables:<br />
<br />
= raw_message table =<br />
<br />
Data received from sending party. <br />
<br />
<table xmlns="http://www.w3.org/1999/xhtml" border=1><br />
<tr><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Name</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Data type</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Not Null?</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Primary key?</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Default</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Comment</th><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">id</td><br />
<td class="ReportTableValueCell">integer</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell">Auto incrementing</td><br />
<td class="ReportTableValueCell"> </td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">message_id</td><br />
<td class="ReportTableValueCell">uuid</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">unique ID of the SMS</td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">sent_from</td><br />
<td class="ReportTableValueCell">character varying</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">number that sent the SMS</td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">body</td><br />
<td class="ReportTableValueCell">character varying</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Body of the SMS</td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">sent_to</td><br />
<td class="ReportTableValueCell">character varying</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">the phone number registered on the SIM card otherwise it's the value set on the app as device ID</td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">gateway_id</td><br />
<td class="ReportTableValueCell">character varying</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">the unique id set on the device to be used by the server to identify which device is communicating with it </td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">sent_timestamp</td><br />
<td class="ReportTableValueCell">timestamp with time zone</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">the timestamp the SMS was sent</td><br />
</tr><br />
</table><br />
<br />
= message table =<br />
<br />
The parsed message body.<br />
<br />
<table xmlns="http://www.w3.org/1999/xhtml" border=1><br />
<tr><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Name</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Data type</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Not Null?</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Primary key?</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Default</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Comment</th><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">id</td><br />
<td class="ReportTableValueCell">integer</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Foreign key raw_message.id</td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">device_info_serial</td><br />
<td class="ReportTableValueCell">integer</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Tracker identifier</td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">date_time</td><br />
<td class="ReportTableValueCell">timestamp with time zone</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Datetime the message was recieved</td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">battery_voltage</td><br />
<td class="ReportTableValueCell">real</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell"> </td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">memory_usage</td><br />
<td class="ReportTableValueCell">real</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell"> </td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">debug_info</td><br />
<td class="ReportTableValueCell">character varying</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell"> </td><br />
</tr><br />
</table><br />
<br />
= position table =<br />
<br />
The positions parsed form the message body.<br />
<br />
<table xmlns="http://www.w3.org/1999/xhtml" border=1><br />
<tr><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Name</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Data type</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Not Null?</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Primary key?</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Default</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Comment</th><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">id</td><br />
<td class="ReportTableValueCell">integer</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell"> </td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">device_info_serial</td><br />
<td class="ReportTableValueCell">integer</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Tracker identifier</td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">date_time</td><br />
<td class="ReportTableValueCell">timestamp with time zone</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Timestamp at which location was taken</td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">lon</td><br />
<td class="ReportTableValueCell">double precision</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Longitude </td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">lat</td><br />
<td class="ReportTableValueCell">double precision</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Latitude</td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">location</td><br />
<td class="ReportTableValueCell">geometry(1107456)</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">PostGIS location as a Point(lon, lat) </td><br />
</tr><br />
</table></div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=SMS_tables&diff=1248SMS tables2015-05-11T11:18:38Z<p>Stefan verhoeven: </p>
<hr />
<div>The eEcology SMS reciever (https://github.com/NLeSC/eEcology-SMS-reciever) service recieves SMS messages and stores them.<br />
<br />
The messages are stored in the PostgreSQL server at db.e-ecology.sara.nl in the eecology database and '''sms''' schema.<br />
The data can be inserted by the eEcology SMS reciever service.<br />
The data can be selected by users with the gps_read role.<br />
<br />
Data is stored in 3 tables:<br />
<br />
= raw_message table =<br />
<br />
Data received from sending party. <br />
<br />
<table xmlns="http://www.w3.org/1999/xhtml" border=1><br />
<tr><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Name</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Data type</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Not Null?</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Primary key?</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Default</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Comment</th><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">id</td><br />
<td class="ReportTableValueCell">integer</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell">Auto incrementing</td><br />
<td class="ReportTableValueCell"> </td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">message_id</td><br />
<td class="ReportTableValueCell">uuid</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">unique ID of the SMS</td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">sent_from</td><br />
<td class="ReportTableValueCell">character varying</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">number that sent the SMS</td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">body</td><br />
<td class="ReportTableValueCell">character varying</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Body of the SMS</td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">sent_to</td><br />
<td class="ReportTableValueCell">character varying</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">the phone number registered on the SIM card otherwise it's the value set on the app as device ID</td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">gateway_id</td><br />
<td class="ReportTableValueCell">character varying</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">the unique id set on the device to be used by the server to identify which device is communicating with it </td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">sent_timestamp</td><br />
<td class="ReportTableValueCell">timestamp with time zone</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">the timestamp the SMS was sent</td><br />
</tr><br />
</table><br />
<br />
= message table =<br />
<br />
The parsed message body.<br />
<br />
<table xmlns="http://www.w3.org/1999/xhtml" border=1><br />
<tr><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Name</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Data type</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Not Null?</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Primary key?</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Default</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Comment</th><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">id</td><br />
<td class="ReportTableValueCell">integer</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Foreign key raw_message.id</td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">device_info_serial</td><br />
<td class="ReportTableValueCell">integer</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Tracker identifier</td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">date_time</td><br />
<td class="ReportTableValueCell">timestamp with time zone</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Datetime the message was recieved</td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">battery_voltage</td><br />
<td class="ReportTableValueCell">real</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell"> </td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">memory_usage</td><br />
<td class="ReportTableValueCell">real</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell"> </td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">debug_info</td><br />
<td class="ReportTableValueCell">character varying</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell"> </td><br />
</tr><br />
</table><br />
<br />
= position table =<br />
<br />
The positions parsed form the message body.<br />
<br />
<table xmlns="http://www.w3.org/1999/xhtml" border=1><br />
<tr><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Name</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Data type</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Not Null?</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Primary key?</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Default</th><br />
<th class="ReportTableHeaderCell" width="16,6666666666667%">Comment</th><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">id</td><br />
<td class="ReportTableValueCell">integer</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell"> </td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">device_info_serial</td><br />
<td class="ReportTableValueCell">integer</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Tracker identifier</td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">date_time</td><br />
<td class="ReportTableValueCell">timestamp with time zone</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell">Yes</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Timestamp at which location was taken</td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">lon</td><br />
<td class="ReportTableValueCell">double precision</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Longitude </td><br />
</tr><br />
<tr class="ReportDetailsEvenDataRow"><br />
<td class="ReportTableValueCell">lat</td><br />
<td class="ReportTableValueCell">double precision</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">Latitude</td><br />
</tr><br />
<tr class="ReportDetailsOddDataRow"><br />
<td class="ReportTableValueCell">location</td><br />
<td class="ReportTableValueCell">geometry(1107456)</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell">No</td><br />
<td class="ReportTableValueCell"> </td><br />
<td class="ReportTableValueCell">PostGIS location as a Point(lon, lat) </td><br />
</tr><br />
</table></div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=SMS_tables&diff=1247SMS tables2015-05-11T10:08:34Z<p>Stefan verhoeven: Created page with "The eEcology SMS reciever (https://github.com/NLeSC/eEcology-SMS-reciever) service recieves SMS messages and stores them. The messages are stored in the PostgreSQL server at db...."</p>
<hr />
<div>The eEcology SMS reciever (https://github.com/NLeSC/eEcology-SMS-reciever) service recieves SMS messages and stores them.<br />
<br />
The messages are stored in the PostgreSQL server at db.e-ecology.sara.nl in the eecology database and '''sms''' schema.<br />
The data can be inserted by the eEcology SMS reciever service.<br />
The data can be selected by users with the gps_read role.<br />
<br />
Data is stored in 3 tables:<br />
* raw_message<br />
* message<br />
* position<br />
<br />
= raw_message table =<br />
<br />
= message table =<br />
<br />
= position table =</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=E-Ecology_Documentation_and_Guidance&diff=1246E-Ecology Documentation and Guidance2015-05-11T10:03:11Z<p>Stefan verhoeven: /* UvA-BiTS Services */</p>
<hr />
<div>The e-Ecology Virtual Laboratories represent a collaborative effort between the University of Amsterdam Institute for Biodiversity and Ecosystem Dynamics and SARA Computing to manage bird tracking data and develop tools for data exploration, analysis and visualization. This wiki is designed to facilitate access to the data and tools within the Virtual Laboratories and to provide a forum for collaboration. <br />
<br />
== UvA-BiTS data == <br />
<br />
:Bird data collected with the UvA-BiTS Bird Tracking System<br />
<br />
Note: We are continually working on improving UvA-BiTS services, including data processing and Virtual Lab functionality. In some cases, this has enabled us to extract more information from old GPS log files which is then incorporated into the database. These changes may result in slight differences when you reprocess historic data. <br />
<br />
::* [[ DB_Views_2015 | e-Ecology database for general use (views) ]]<br />
::* [[ DB_Tables_2015 | e-Ecology tables behind the views (not accessible for normal use) ]]<br />
::* [[ UvAGPS|UvA-BiTS Data Model]] - OLD<br />
::* [[Connecting to the UvA-BiTS database]]<br />
::* [[Useful Queries]]<br />
::* [[Queries for linking bird tracks to wind and flow data]]<br />
<br />
==UvA-BiTS Services==<br />
<br />
* Visualization Tools<br />
:: [[BirdView]]: <br /><br />
:::Access meterological and radar data and overlay with GPS bird tracking data.<br />
:: [[BirdSim]]: <br/ ><br />
:::More dynamic version of BirdView that allows the user to scroll through weather data in time and space.<br />
:: [[KMZ Generator|UvA-BiTS KMZ Generator]]: <br /><br />
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth. <br />
:: [[Tracker Calendar]]: <br /><br />
:::Calendar overview with daily statistics of GPS-tracker<br />
:: [[Flight Generator]]: <br /><br />
:::This tool produces GPX files from UvA-BiTS tracking data for dynamic [http://www.doarama.com Doarama] visualizations.<br />
<br />
* [[Dashboard]]: These web pages allow UvA-BiTS users to get an overview of the raw data files of GPS bird tracking devices they have access to and to access the dashboard for administering projects. <br />
<br />
* [[ Annotation tables ]] for uploading GPS annotation data into your own schema. This service is currently not available.<br />
* [[ SMS tables ]] for storage of SMS messages send by trackers.<br />
<br />
== Changes in the Database in 2014 ==<br />
'''Important'''<br />
In the summer of 2014 we have '''re-arranged the administrative''' part of UvA-BiTS (and e-Ecology) in the database. End users may have to adapt their programs the following way:<br />
:* The '''uva_*_limited''' views that used to exist in the gps schema are gone. Most of them should have been replaced with an equivalent '''ee_*_limited counterpart'''. <br />
:* The new ee_*_limited views show a superset of the same information as before, but enhanced with new project information.<br />
:* Some '''column names''' in the ee_*_limited views may differ from the previous views.<br />
:* There are new limited views: <br />
:** gps.ee_'''project'''_limited: shows your projects' information<br />
:** gps.ee_'''shared'''_*_limited: set of data you are able to see related to trackers that a project is sharing with a project you belong to<br />
:* Special attention: gps.uva_device_limited is now replaced by '''gps.ee_tracker_limited'''. For each tracker, you now see which project provides you access to that tracker.<br />
:* Also, '''gps.uva_tracking_limited''' is gone. Instead of that we have gps.ee_tracker_speed_limited, which adds speed columns to the original data. This amounts to a merger of gps.ee_tracking_limited, gps.ee_tracking_speed_3d_limited and gps.ee_tracking_speed_limited.<br />
<br />
==Other e-Ecology Datasets==<br />
<br />
'''ENVIRONMENTAL DATA'''<br />
<br />
::* [[ ECMWF|Weather Forecast Data (ECMWF)]]<br />
::* [[ Elevation|Elevation Data ]]<br />
<br />
'''FLYSAFE'''<br />
<br />
:Bird data created from weather radar data and bird migration models. Financed by the Royal Netherlands Air Force. <br />
<br />
::* [[Connecting to the Flysafe database]]<br />
::* [[GPS|Flysafe GPS Data Model]]<br />
::* [[ RADAR|Radar Data Model]]<br />
::* [[ Wind profile|Wind Profile Data Model]]<br />
::* [[ Fino1|Fino1 Data Model]]<br />
<br />
==Other e-Ecology Services==<br />
[[GPS tracks|FlySafe GPS Bird Track Generator]]: Produce kmz files of ARGOS/GPS bird tracking data from the FlySafe project to be viewed in Google Earth. <br />
<br />
[[ Administration]]: Information about user rights and data processing tasks.<br />
<br />
[[ Bookkeeping]]: Description of tables related to FlySafe data processing and archiving.<br />
<br />
[[ ToDo ]]</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=E-Ecology_Documentation_and_Guidance&diff=1228E-Ecology Documentation and Guidance2015-01-13T16:56:46Z<p>Stefan verhoeven: /* UvA-BiTS Services */</p>
<hr />
<div>The e-Ecology Virtual Laboratories represent a collaborative effort between the University of Amsterdam Institute for Biodiversity and Ecosystem Dynamics and SARA Computing to manage bird tracking data and develop tools for data exploration, analysis and visualization. This wiki is designed to facilitate access to the data and tools within the Virtual Laboratories and to provide a forum for collaboration. <br />
<br />
== UvA-BiTS data == <br />
<br />
:Bird data collected with the UvA-BiTS Bird Tracking System<br />
<br />
Note: We are continually working on improving UvA-BiTS services, including data processing and Virtual Lab functionality. In some cases, this has enabled us to extract more information from old GPS log files which is then incorporated into the database. These changes may result in slight differences when you reprocess historic data. <br />
<br />
::* [[ DB_Views_2015 | e-Ecology database for general use (views) ]]<br />
::* [[ DB_Tables_2015 | e-Ecology tables behind the views (not accessible for normal use) ]]<br />
::* [[ UvAGPS|UvA-BiTS Data Model]] - OLD<br />
::* [[Connecting to the UvA-BiTS database]]<br />
::* [[Useful Queries]]<br />
::* [[Queries for linking bird tracks to wind and flow data]]<br />
<br />
==UvA-BiTS Services==<br />
<br />
* Visualization Tools<br />
:: [[BirdView]]: <br /><br />
:::Access meterological and radar data and overlay with GPS bird tracking data.<br />
:: [[BirdSim]]: <br/ ><br />
:::More dynamic version of BirdView that allows the user to scroll through weather data in time and space.<br />
:: [[KMZ Generator|UvA-BiTS KMZ Generator]]: <br /><br />
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth. <br />
:: [[Tracker Calendar]]: <br /><br />
:::Calendar overview with daily statistics of GPS-tracker<br />
:: [[Flight Generator]]: <br /><br />
:::This tool produces GPX files from UvA-BiTS tracking data for dynamic [http://www.doarama.com Doarama] visualizations.<br />
<br />
* [[Dashboard]]: These web pages allow UvA-BiTS users to get an overview of the raw data files of GPS bird tracking devices.<br />
<br />
* [[ Annotation tables ]] for uploading GPS annotation data into your own schema. This service is currently not available.<br />
<br />
== Changes in the Database in 2014 ==<br />
'''Important'''<br />
In the summer of 2014 we have '''re-arranged the administrative''' part of UvA-BiTS (and e-Ecology) in the database. End users may have to adapt their programs the following way:<br />
:* The '''uva_*_limited''' views that used to exist in the gps schema are gone. Most of them should have been replaced with an equivalent '''ee_*_limited counterpart'''. <br />
:* The new ee_*_limited views show a superset of the same information as before, but enhanced with new project information.<br />
:* Some '''column names''' in the ee_*_limited views may differ from the previous views.<br />
:* There are new limited views: <br />
:** gps.ee_'''project'''_limited: shows your projects' information<br />
:** gps.ee_'''shared'''_*_limited: set of data you are able to see related to trackers that a project is sharing with a project you belong to<br />
:* Special attention: gps.uva_device_limited is now replaced by '''gps.ee_tracker_limited'''. For each tracker, you now see which project provides you access to that tracker.<br />
:* Also, '''gps.uva_tracking_limited''' is gone. Instead of that we have gps.ee_tracker_speed_limited, which adds speed columns to the original data. This amounts to a merger of gps.ee_tracking_limited, gps.ee_tracking_speed_3d_limited and gps.ee_tracking_speed_limited.<br />
<br />
==Other e-Ecology Datasets==<br />
<br />
'''ENVIRONMENTAL DATA'''<br />
<br />
::* [[ ECMWF|Weather Forecast Data (ECMWF)]]<br />
::* [[ Elevation|Elevation Data ]]<br />
<br />
'''FLYSAFE'''<br />
<br />
:Bird data created from weather radar data and bird migration models. Financed by the Royal Netherlands Air Force. <br />
<br />
::* [[Connecting to the Flysafe database]]<br />
::* [[GPS|Flysafe GPS Data Model]]<br />
::* [[ RADAR|Radar Data Model]]<br />
::* [[ Wind profile|Wind Profile Data Model]]<br />
::* [[ Fino1|Fino1 Data Model]]<br />
<br />
==Other e-Ecology Services==<br />
[[GPS tracks|FlySafe GPS Bird Track Generator]]: Produce kmz files of ARGOS/GPS bird tracking data from the FlySafe project to be viewed in Google Earth. <br />
<br />
[[ Administration]]: Information about user rights and data processing tasks.<br />
<br />
[[ Bookkeeping]]: Description of tables related to FlySafe data processing and archiving.<br />
<br />
[[ ToDo ]]</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=KMZ_Generator&diff=1227KMZ Generator2015-01-13T16:56:18Z<p>Stefan verhoeven: </p>
<hr />
<div>== The UvA-BiTS KMZ Generator==<br />
<br />
The UvA-BiTS GPS tracking data can be overlaid on maps in a broad range of software packages. This track generator creates a [http://en.wikipedia.org/wiki/Keyhole_Markup_Language KMZ] file which can be loaded into Google Earth, as well as many other GIS-systems which provide KMZ reading capabilities.<br />
<br />
The service itself can be found via the Virtual Lab [http://www.uva-bits.nl/virtual-lab/] by selecting "kmz generator". This requires special access: You need to have credentials and your IP address should be registered. Please contact [mailto://UvA-BiTS@uva.nl UvA-BiTS@uva.nl] if you have problems getting access.<br />
<br />
=== System Requirements ===<br />
<br />
==== Software Requirements ====<br />
The webservice itself is standard javascript website, and as such is supported by all modern browsers and operating systems.<br />
<br />
For looking at KML files, there are multiple options. The option we support is [http://earth.google.com/ Google Earth] .<br />
<br />
==== Hardware requirements ====<br />
[http://earth.google.com/ Google Earth] is a visual applications, and therefore do require a little bit of graphical power. Fortunately, most modern computers have decent hardware for graphical acceleration, so this should not be a problem. But for example, if the graphical drivers are not installed correctly, or if remote desktop tools are used, performance can be limited. <br />
<br />
Additionally, [http://earth.google.com/ Google Earth] require a decent internet connection. They use satellite images as background, and these have to be retrieved from the internet. Furthermore, if large amounts of data are selected in the kmz generator, performance issues may arise.<br />
<br />
=== The Interface ===<br />
[[Image:Swkmzgen.png|thumb|right|400px|abc|]]<br />
<br />
In order to be able to select specific bird(s) and timespan(s) for detailed studying, an interface was built that allows users to query and retrieve specified subsets of the available data. See the image to the right for a screenshot of the interface.<br />
<br />
To use the KMZ generator:<br />
# Select start date and end date<br />
# Select trackers<br />
# Customize tracker color<br />
# Advanced settings (optionally)<br />
# Save selection (optionally)<br />
# Press submit button<br />
# Wait for kmz to be generated<br />
# Download kmz and open it Google Earth<br />
<br />
==== Select start date and end date ====<br />
<br />
For the begin and end date, there are two options: You can either type a date directly, or you can use the calendar button. When you use the calendar button, a small window pops up where a desired date can be selected. Here you can either use the arrows, or you can use ctrl up/down and ctrl left/right to change the years and months respectively. If you have selected a year and month, you can click on a day to finalize you date selection. For more detailed time tuning, there are also fields for the hours and minutes. Again, you can either type it, or you can use the drop down to select the desired hour.<br />
<br />
==== Select trackers ====<br />
<br />
To select trackers they must be moved from the 'Available' table to the 'Selected' table. Moving tracker can be done in 3 ways:<br />
* double click on tracker<br />
* drag and drop<br />
* select it and press arrow button<br />
Multiple trackers can be selected at the same time.<br />
<br />
==== Customize tracker color ====<br />
<br />
The selected trackers are automatically assigned colors. The colors can be changed by clicking on the colored bar or pencil icon. A list of colors will be shown, clicking will select it.<br />
<br />
==== Advanced settings ====<br />
<br />
The advanced settings can be opened by clicking on the ''Advanced settings'' text on between the tracker selection and submit buttons.<br />
Available settings:<br />
* Shape, use a circle icon or an directional arrow. The direction can be based on instantaneous direction (the direction the GPS tracker reported) or traject direction (direction between current point and previous point)<br />
* Size, use small, medium or large icons<br />
* Size based on altitude, when checked the icons will be bigger when altitude is higher.<br />
* Color, can be single color or a gradient. Which color gradiention to use is based on the speed. The higher the speed the darker. Same as directional arrow there is a choice between instantaneous and trajectory speed.<br />
* Speed thresholds, when speed gradient is selected for color then you can choose at which speed which color should be used.<br />
* Transparency, when you have many points or lines in the same place transparency can be used to make density map by making the points and lines more transparent.<br />
* Altitude, the altitude of points can be configure to be: <br />
** absolute, will use the altitude of the tracker<br />
** clamp to ground, will snap the point to the ground<br />
** relative to ground, the ground elevation will be subtracted from the altitude of the tracker<br />
<br />
==== Save selection ====<br />
<br />
Users often are interested in the same trackers. It would be tedious to select the same trackers each time. Therefore we added functionality to save and load specific selections.<br />
To save the current selection, press the ''Save selection'' button, choose a name, by default the current time will be used.<br />
To restore a saved selection, press the ''Restored saved selection'' button, double click on the name or press the icon with the arrow.<br />
<br />
As a bonus the last submitted selection is always saved as ''Last used''. This can be used when you want to make the same kml file again or make small changes without configuring everything again.<br />
<br />
It must be noted that the saved selections are browser specific. So will not show up when using a different computer.<br />
<br />
=== Limitations ===<br />
<br />
The KMZ generator will give a error when you try to make a KMZ file with more than 50.000 datapoints in total.<br />
The error will tell you how many data point are currently selected.<br />
<br />
===Visualization ===<br />
[[Image:GPS_visualization_screenshot1.png|thumb|right|560px|GPS track Visualization in Google Earth|]]<br />
<br />
The KMZ generator generates KMZ files containing one or more trackers. Currently, it saves the resulting file as t<trackers>-s<starttime>-e<endtime>.kmz , where <trackers> is a list of tracker identifiers separated by and ''_'', <starttime> is the start time in yyyymmddhhmm format and <endtime> is the endtime in yyyymmddhhmm format.<br />
The kmz itself can be viewed with a variety of applications, provided they support the KML format of files. In this documentation, we will limit ourselves to the use of Google Earth.<br />
<br />
==== Data Representation ====<br />
<br />
The data of each bird track consists of two different parts. The first part is a simple line of the chosen color showing the route that the bird has flown between the selected dates. The route is created by connecting all the GPS measurements in order of their timestamps. This gives a clear view of foraging and migratory habits over longer periods of time.<br />
<br />
The second part is a collection of icons, each one representing a GPS tag measurement on a certain time-step, and located on the latitude/longitude of the measurement. Each icon has several different visual indications of the measurement, such as:<br />
* The size of the icon is an indication of the altitude of the measurement, where the bigger icon represents a higher altitude.<br />
* The color of the icon is an indication of speed<br />
<br />
Because one bird track can consist of a large number of measurements, which are sometimes located in a small area.<br />
* Because each measurement has a timestamp attached to it, you can use the built in Google Earth time slider to reduce the number of measurements visible. <br />
<br />
=== Underlying Query and Data ===<br />
<br />
The following sql query is used:<br />
<br />
SELECT <br />
s.device_info_serial <br />
s.date_time, <br />
s.longitude, <br />
s.latitude, <br />
s.altitude, <br />
s.altitude_agl,<br />
round(CAST(s.speed_2d AS NUMERIC), 2) AS ispeed, <br />
round(CAST(ST_Length_Spheroid(ST_MakeLine(s.location, lag(s.location) OVER (ORDER BY s.device_info_serial, s.date_time)), 'SPHEROID["WGS 84",6378137,298.257223563]') / EXTRACT(epoch FROM s.date_time - lag(s.date_time) OVER (ORDER BY s.device_info_serial, s.date_time)) AS NUMERIC), 2) AS tspeed, <br />
round(s.direction, 2) AS idirection, <br />
round(CAST(degrees(ST_Azimuth(lag(s.location) OVER (ORDER BY s.device_info_serial, s.date_time), s.location)) AS NUMERIC), 2) AS tdirection,<br />
FROM gps.ee_tracking_speed_limited s<br />
WHERE <br />
s.device_info_serial = 583<br />
AND s.date_time BETWEEN '2013-04-01' AND '2013-06-01'<br />
AND s.longitude IS NOT NULL <br />
AND s.latitude IS NOT NULL <br />
AND s.userflag = 0<br />
ORDER BY s.date_time<br />
;</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=KMZ_Generator&diff=1226KMZ Generator2015-01-13T16:52:19Z<p>Stefan verhoeven: moved KMZ Generator to KMZ Generator Old: Has been replaced by new kmz generator</p>
<hr />
<div>#REDIRECT [[KMZ Generator Old]]</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=KMZ_Generator_Old&diff=1225KMZ Generator Old2015-01-13T16:52:19Z<p>Stefan verhoeven: moved KMZ Generator to KMZ Generator Old: Has been replaced by new kmz generator</p>
<hr />
<div>== The UvA-BiTS KMZ Generator==<br />
<br />
The UvA-BiTS GPS tracking data can be overlaid on maps in a broad range of software packages. This track generator creates a [http://en.wikipedia.org/wiki/Keyhole_Markup_Language KMZ] file which can be loaded into Google Earth, as well as many other GIS-systems which provide KMZ reading capabilities.<br />
<br />
The service itself can be found via the Virtual Lab [http://www.uva-bits.nl/virtual-lab/] by selecting "kmz generator" or via direct link at [https://public.e-ecology.sara.nl/uvagpstrackingext.html/ e-ecology website]. This requires special access: You need to have credentials and your IP address should be registered. Please contact grid.support@sara.nl if you have problems getting access.<br />
<br />
=== System Requirements ===<br />
<br />
==== Software Requirements ====<br />
The webservice itself is standard javascript website, and as such is supported by all modern browsers and operating systems.<br />
<br />
For looking at KML files, there are multiple options. The options we support are [http://earth.google.com/ Google Earth] and [http://public.flysafe.sara.nl/birdview.html Birdview].<br />
<br />
==== Hardware requirements ====<br />
Both [http://earth.google.com/ Google Earth] and [http://birdvis.e-ecology.sara.nl/birdview.html Birdview] are visual applications, and therefore do require a little bit of graphical power. Fortunately, most modern computers have decent hardware for graphical acceleration, so this should not be a problem. But for example, if the graphical drivers are not installed correctly, or if remote desktop tools are used, performance can be limited. <br />
<br />
Additionally, both [http://earth.google.com/ Google Earth] and [http://birdvis.e-ecology.sara.nl/birdview.html Birdview] require a decent internet connection. They use satellite images as background, and these have to be retrieved from the internet. Furthermore, if large amounts of data are selected in the kmz generator, performance issues may arise.<br />
<br />
=== The Interface ===<br />
[[Image:GPS_interface_screenshot1.png|thumb|right|400px|abc|]]<br />
<br />
Records exist from May 2008 and onwards.<br />
In order to be able to select specific bird(s) and timespan(s) for detailed studying, an interface was built that allows users to query and retrieve specified subsets of the available data. See the image to the right for a screenshot of the interface.<br />
<br />
To use the GPS service, one must first make one or more bird selections, where "add new" button is used to add a selection to the list.<br />
When the user is satisfied with the list, the submit button can be pressed to generate the KMZ file with all the selected data.<br />
<br />
Each individual option in the interface is described in more detail below.<br />
<br />
==== Bird and Time selection ====<br />
<br />
The two drop down menu's in the upper left of the interface are for selection of specific birds. The species drop down, when used, filters the available birds so that only birds of the selected species are visible. The device-id drop down shows all the unique bird ID's. Every single number is a unique bird. <br />
<br />
In the upper right of the interface there are selectors to specify the time range. For the begin and end date, there are two options: You can either type a date directly, or you can use the calender button. When you use the calender button, a small window pops up where a desired date can be selected. Here you can either use the arrows, or you can use ctrl up/down and ctrl left/right to change the years and months respectively. If you have selected a year and month, you can click on a day to finalize you date selection.<br />
For more detailed time tuning, there are also fields for the hours and minutes. Again, you can either type it, or you can use the drop down to select the desired hour.<br />
<br />
==== Visualization and calculation options ====<br />
<br />
There are a few options in the interface that modify the visual properties of the bird tracks:<br />
* The Color option assigns a specific color to the selected bird track. Here there are two version or each color: ''normal'' and ''range''. In the ''normal option, all icons are the same color, but with the ''range'' option, the icons are colored according to speed, where the darker shades correspond to lower speeds, while the whiter shades correspond to higher speeds.<br />
* The icon option lets the user choose between two different icons, the circle and the arrow. The difference is that the arrow shows direction, while the circle does not.<br />
* The ''Force calculated speed/heading'' checkbox is an option that forces the underlying scripts to calculate speed and heading instead of using the measured speed and heading in the database. The reason for this is that not all GPS tags had the capability for measuring speed and heading. With this option we provide a homogenized speed/heading option for all type of tags. The calculated speed is the distance divided by the time since last measurement, the calculated heading is the heading of the shortest route from last measurement.<br />
More details about the visualization can be found in [[#Visualization]].<br />
<br />
==== Selection list and Submitting ====<br />
<br />
When you are finished selecting a bird, a timespan and your visualization options, you can add the bird to you current selection list with the "Add New" button. You then have the option of either adding more birds, or if you're satisfied with your selection, you can submit your selection to the server with the "Submit" button. After submitting, you have to wait a bit (depending on the size of your selection) while the server queries the database and constructs your KML file. <br />
<br />
Because users often have specific bird(s) that they are interested in, it would be tedious to fill in the form with the same selections every time. Therefore we have added functionality to save and load specific selections. To save the current selection, simply press the ''Save Selection'' button, choose a name, and press Save. <br />
To load a previously saved selection, simply press the '' Previous Selections'' button, select the list your interested in, and press Load.<br />
<br />
It must be noted that the selection are saved as cookies, and are therefore browser specific, and can be lost over time.<br />
<br />
===Visualization ===<br />
[[Image:GPS_visualization_screenshot1.png|thumb|right|560px|GPS track Visualization in Google Earth|]]<br />
<br />
The GPS service generates KML files containing one or more bird tracks. Currently, it saves the resulting file as tracks<#1>-<#2>.kml, where <#1> is the lowest bird ID, and <#2> the highest bird ID in the file.<br />
The kml itself can be viewed with a variety of applications, provided they support the KML format of files. In this documentation, we will limit ourselves to the use of Google Earth.<br />
<br />
==== Data Representation ====<br />
<br />
The data of each bird track consists of two different parts. The first part is a simple line of the chosen color showing the route that the bird has flown between the selected dates. The route is created by connecting all the GPS measurements in order of their timestamps. This gives a clear view of foraging and migratory habits over longer periods of time.<br />
<br />
The second part is a collection of icons, each one representing a GPS tag measurement on a certain time-step, and located on the latitude/longitude of the measurement. Each icon has several different visual indications of the measurement, such as:<br />
* The size of the icon is an indication of the altitude of the measurement, where the bigger icon represents a higher altitude.<br />
* The color of the icon is an indication<br />
<br />
<br />
<br />
Because one bird track can consist of a large number of measurements, which are sometimes located in a small area, there are several tricks that can be used when looking at the data in Google earth.<br />
* Because each measurement has a timestamp attached to it, you can use the built in Google Earth time slider to specify <br />
<br />
=== Underlying Query and Data ===<br />
<br />
The data in the database is produced with different models of GPS trackers. The result is that not every bird has exact the same data available. However, in order to provide compatibility, they are all put in the same table, but with the older models having certain fields empty. The following query is used:<br />
<br />
SELECT g.ptt_id, i.species, g.date_time, g.latitude, g.longitude, g.speed, g.course, g.altitude,<br />
g.alt_status, g.quality, g.user_qc, g.parser_qc, g."location", elevation.srtm_getvalue(g."location") <br />
AS elevation, case when g.altitude = 0 then NULL else (g.altitude - elevation.srtm_getvalue(g."location")) end AS agl <br />
FROM gps.gps_tracking_data g, gps.individual i, gps.track_session t <br />
WHERE g.ptt_id = t.ptt_id AND g.date_time >= t.start_date <br />
AND i.ring_number = t.ring_number AND g.parser_qc = 1 AND g.ptt_id = ''<ptt_id>''<br />
AND g.date_time >= ''<startTime>'' AND g.date_time <= ''<endTime>'' <br />
AND g.latitude <> 0 ORDER BY g.date_time<br />
<br />
This query returns a table of tracks, which then if filtered so that all entries with either 'quality' =="FAILED"' or 'parser_qc' >1' are removed, because these measurements were considered errors</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=File:CalendarSubmit.png&diff=1223File:CalendarSubmit.png2015-01-12T10:53:25Z<p>Stefan verhoeven: uploaded a new version of &quot;File:CalendarSubmit.png&quot;</p>
<hr />
<div>Tracker calendar submit screenshot</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=File:GPXSubmit.png&diff=1222File:GPXSubmit.png2015-01-12T10:43:45Z<p>Stefan verhoeven: uploaded a new version of &quot;File:GPXSubmit.png&quot;</p>
<hr />
<div>Doarama generator/flight generator/ gpx file submit form</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=KMZ_Generator2&diff=1221KMZ Generator22015-01-12T10:37:55Z<p>Stefan verhoeven: </p>
<hr />
<div>== The UvA-BiTS KMZ Generator==<br />
<br />
The UvA-BiTS GPS tracking data can be overlaid on maps in a broad range of software packages. This track generator creates a [http://en.wikipedia.org/wiki/Keyhole_Markup_Language KMZ] file which can be loaded into Google Earth, as well as many other GIS-systems which provide KMZ reading capabilities.<br />
<br />
The service itself can be found via the Virtual Lab [http://www.uva-bits.nl/virtual-lab/] by selecting "kmz generator". This requires special access: You need to have credentials and your IP address should be registered. Please contact [mailto://UvA-BiTS@uva.nl UvA-BiTS@uva.nl] if you have problems getting access.<br />
<br />
=== System Requirements ===<br />
<br />
==== Software Requirements ====<br />
The webservice itself is standard javascript website, and as such is supported by all modern browsers and operating systems.<br />
<br />
For looking at KML files, there are multiple options. The option we support is [http://earth.google.com/ Google Earth] .<br />
<br />
==== Hardware requirements ====<br />
[http://earth.google.com/ Google Earth] is a visual applications, and therefore do require a little bit of graphical power. Fortunately, most modern computers have decent hardware for graphical acceleration, so this should not be a problem. But for example, if the graphical drivers are not installed correctly, or if remote desktop tools are used, performance can be limited. <br />
<br />
Additionally, [http://earth.google.com/ Google Earth] require a decent internet connection. They use satellite images as background, and these have to be retrieved from the internet. Furthermore, if large amounts of data are selected in the kmz generator, performance issues may arise.<br />
<br />
=== The Interface ===<br />
[[Image:Swkmzgen.png|thumb|right|400px|abc|]]<br />
<br />
In order to be able to select specific bird(s) and timespan(s) for detailed studying, an interface was built that allows users to query and retrieve specified subsets of the available data. See the image to the right for a screenshot of the interface.<br />
<br />
To use the KMZ generator:<br />
# Select start date and end date<br />
# Select trackers<br />
# Customize tracker color<br />
# Advanced settings (optionally)<br />
# Save selection (optionally)<br />
# Press submit button<br />
# Wait for kmz to be generated<br />
# Download kmz and open it Google Earth<br />
<br />
==== Select start date and end date ====<br />
<br />
For the begin and end date, there are two options: You can either type a date directly, or you can use the calendar button. When you use the calendar button, a small window pops up where a desired date can be selected. Here you can either use the arrows, or you can use ctrl up/down and ctrl left/right to change the years and months respectively. If you have selected a year and month, you can click on a day to finalize you date selection. For more detailed time tuning, there are also fields for the hours and minutes. Again, you can either type it, or you can use the drop down to select the desired hour.<br />
<br />
==== Select trackers ====<br />
<br />
To select trackers they must be moved from the 'Available' table to the 'Selected' table. Moving tracker can be done in 3 ways:<br />
* double click on tracker<br />
* drag and drop<br />
* select it and press arrow button<br />
Multiple trackers can be selected at the same time.<br />
<br />
==== Customize tracker color ====<br />
<br />
The selected trackers are automatically assigned colors. The colors can be changed by clicking on the colored bar or pencil icon. A list of colors will be shown, clicking will select it.<br />
<br />
==== Advanced settings ====<br />
<br />
The advanced settings can be opened by clicking on the ''Advanced settings'' text on between the tracker selection and submit buttons.<br />
Available settings:<br />
* Shape, use a circle icon or an directional arrow. The direction can be based on instantaneous direction (the direction the GPS tracker reported) or traject direction (direction between current point and previous point)<br />
* Size, use small, medium or large icons<br />
* Size based on altitude, when checked the icons will be bigger when altitude is higher.<br />
* Color, can be single color or a gradient. Which color gradiention to use is based on the speed. The higher the speed the darker. Same as directional arrow there is a choice between instantaneous and trajectory speed.<br />
* Speed thresholds, when speed gradient is selected for color then you can choose at which speed which color should be used.<br />
* Transparency, when you have many points or lines in the same place transparency can be used to make density map by making the points and lines more transparent.<br />
* Altitude, the altitude of points can be configure to be: <br />
** absolute, will use the altitude of the tracker<br />
** clamp to ground, will snap the point to the ground<br />
** relative to ground, the ground elevation will be subtracted from the altitude of the tracker<br />
<br />
==== Save selection ====<br />
<br />
Users often are interested in the same trackers. It would be tedious to select the same trackers each time. Therefore we added functionality to save and load specific selections.<br />
To save the current selection, press the ''Save selection'' button, choose a name, by default the current time will be used.<br />
To restore a saved selection, press the ''Restored saved selection'' button, double click on the name or press the icon with the arrow.<br />
<br />
As a bonus the last submitted selection is always saved as ''Last used''. This can be used when you want to make the same kml file again or make small changes without configuring everything again.<br />
<br />
It must be noted that the saved selections are browser specific. So will not show up when using a different computer.<br />
<br />
=== Limitations ===<br />
<br />
The KMZ generator will give a error when you try to make a KMZ file with more than 50.000 datapoints in total.<br />
The error will tell you how many data point are currently selected.<br />
<br />
===Visualization ===<br />
[[Image:GPS_visualization_screenshot1.png|thumb|right|560px|GPS track Visualization in Google Earth|]]<br />
<br />
The KMZ generator generates KMZ files containing one or more trackers. Currently, it saves the resulting file as t<trackers>-s<starttime>-e<endtime>.kmz , where <trackers> is a list of tracker identifiers separated by and ''_'', <starttime> is the start time in yyyymmddhhmm format and <endtime> is the endtime in yyyymmddhhmm format.<br />
The kmz itself can be viewed with a variety of applications, provided they support the KML format of files. In this documentation, we will limit ourselves to the use of Google Earth.<br />
<br />
==== Data Representation ====<br />
<br />
The data of each bird track consists of two different parts. The first part is a simple line of the chosen color showing the route that the bird has flown between the selected dates. The route is created by connecting all the GPS measurements in order of their timestamps. This gives a clear view of foraging and migratory habits over longer periods of time.<br />
<br />
The second part is a collection of icons, each one representing a GPS tag measurement on a certain time-step, and located on the latitude/longitude of the measurement. Each icon has several different visual indications of the measurement, such as:<br />
* The size of the icon is an indication of the altitude of the measurement, where the bigger icon represents a higher altitude.<br />
* The color of the icon is an indication of speed<br />
<br />
Because one bird track can consist of a large number of measurements, which are sometimes located in a small area.<br />
* Because each measurement has a timestamp attached to it, you can use the built in Google Earth time slider to reduce the number of measurements visible. <br />
<br />
=== Underlying Query and Data ===<br />
<br />
The following sql query is used:<br />
<br />
SELECT <br />
s.device_info_serial <br />
s.date_time, <br />
s.longitude, <br />
s.latitude, <br />
s.altitude, <br />
s.altitude_agl,<br />
round(CAST(s.speed_2d AS NUMERIC), 2) AS ispeed, <br />
round(CAST(ST_Length_Spheroid(ST_MakeLine(s.location, lag(s.location) OVER (ORDER BY s.device_info_serial, s.date_time)), 'SPHEROID["WGS 84",6378137,298.257223563]') / EXTRACT(epoch FROM s.date_time - lag(s.date_time) OVER (ORDER BY s.device_info_serial, s.date_time)) AS NUMERIC), 2) AS tspeed, <br />
round(s.direction, 2) AS idirection, <br />
round(CAST(degrees(ST_Azimuth(lag(s.location) OVER (ORDER BY s.device_info_serial, s.date_time), s.location)) AS NUMERIC), 2) AS tdirection,<br />
FROM gps.ee_tracking_speed_limited s<br />
WHERE <br />
s.device_info_serial = 583<br />
AND s.date_time BETWEEN '2013-04-01' AND '2013-06-01'<br />
AND s.longitude IS NOT NULL <br />
AND s.latitude IS NOT NULL <br />
AND s.userflag = 0<br />
ORDER BY s.date_time<br />
;</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=KMZ_Generator2&diff=1220KMZ Generator22015-01-12T10:34:45Z<p>Stefan verhoeven: /* Software Requirements */</p>
<hr />
<div>== The UvA-BiTS KMZ Generator==<br />
<br />
The UvA-BiTS GPS tracking data can be overlaid on maps in a broad range of software packages. This track generator creates a [http://en.wikipedia.org/wiki/Keyhole_Markup_Language KMZ] file which can be loaded into Google Earth, as well as many other GIS-systems which provide KMZ reading capabilities.<br />
<br />
The service itself can be found via the Virtual Lab [http://www.uva-bits.nl/virtual-lab/] by selecting "kmz generator". This requires special access: You need to have credentials and your IP address should be registered. Please contact [mailto://UvA-BiTS@uva.nl UvA-BiTS@uva.nl] if you have problems getting access.<br />
<br />
=== System Requirements ===<br />
<br />
==== Software Requirements ====<br />
The webservice itself is standard javascript website, and as such is supported by all modern browsers and operating systems.<br />
<br />
For looking at KML files, there are multiple options. The option we support is [http://earth.google.com/ Google Earth] .<br />
<br />
==== Hardware requirements ====<br />
Both [http://earth.google.com/ Google Earth] and [http://birdvis.e-ecology.sara.nl/birdview.html Birdview] are visual applications, and therefore do require a little bit of graphical power. Fortunately, most modern computers have decent hardware for graphical acceleration, so this should not be a problem. But for example, if the graphical drivers are not installed correctly, or if remote desktop tools are used, performance can be limited. <br />
<br />
Additionally, both [http://earth.google.com/ Google Earth] and [http://birdvis.e-ecology.sara.nl/birdview.html Birdview] require a decent internet connection. They use satellite images as background, and these have to be retrieved from the internet. Furthermore, if large amounts of data are selected in the kmz generator, performance issues may arise.<br />
<br />
=== The Interface ===<br />
[[Image:Swkmzgen.png|thumb|right|400px|abc|]]<br />
<br />
Records exist from May 2008 and onwards.<br />
In order to be able to select specific bird(s) and timespan(s) for detailed studying, an interface was built that allows users to query and retrieve specified subsets of the available data. See the image to the right for a screenshot of the interface.<br />
<br />
To use the KMZ generator:<br />
# Select start date and end date<br />
# Select trackers<br />
# Customize tracker color<br />
# Advanced settings (optionally)<br />
# Save selection (optionally)<br />
# Press submit button<br />
# Wait for kmz to be generated<br />
# Download kmz and open it Google Earth<br />
<br />
==== Select start date and end date ====<br />
<br />
For the begin and end date, there are two options: You can either type a date directly, or you can use the calendar button. When you use the calendar button, a small window pops up where a desired date can be selected. Here you can either use the arrows, or you can use ctrl up/down and ctrl left/right to change the years and months respectively. If you have selected a year and month, you can click on a day to finalize you date selection. For more detailed time tuning, there are also fields for the hours and minutes. Again, you can either type it, or you can use the drop down to select the desired hour.<br />
<br />
==== Select trackers ====<br />
<br />
To select trackers they must be moved from the 'Available' table to the 'Selected' table. Moving tracker can be done in 3 ways:<br />
* double click on tracker<br />
* drag and drop<br />
* select it and press arrow button<br />
Multiple trackers can be selected at the same time.<br />
<br />
==== Customize tracker color ====<br />
<br />
The selected trackers are automatically assigned colors. The colors can be changed by clicking on the colored bar or pencil icon. A list of colors will be shown, clicking will select it.<br />
<br />
==== Advanced settings ====<br />
<br />
The advanced settings can be opened by clicking on the ''Advanced settings'' text on between the tracker selection and submit buttons.<br />
Available settings:<br />
* Shape, use a circle icon or an directional arrow. The direction can be based on instantaneous direction (the direction the GPS tracker reported) or traject direction (direction between current point and previous point)<br />
* Size, use small, medium or large icons<br />
* Size based on altitude, when checked the icons will be bigger when altitude is higher.<br />
* Color, can be single color or a gradient. Which color gradiention to use is based on the speed. The higher the speed the darker. Same as directional arrow there is a choice between instantaneous and trajectory speed.<br />
* Speed thresholds, when speed gradient is selected for color then you can choose at which speed which color should be used.<br />
* Transparency, when you have many points or lines in the same place transparency can be used to make density map by making the points and lines more transparent.<br />
* Altitude, the altitude of points can be configure to be: <br />
** absolute, will use the altitude of the tracker<br />
** clamp to ground, will snap the point to the ground<br />
** relative to ground, the ground elevation will be subtracted from the altitude of the tracker<br />
<br />
==== Save selection ====<br />
<br />
Users often are interested in the same trackers. It would be tedious to select the same trackers each time. Therefore we added functionality to save and load specific selections.<br />
To save the current selection, press the ''Save selection'' button, choose a name, by default the current time will be used.<br />
To restore a saved selection, press the ''Restored saved selection'' button, double click on the name or press the icon with the arrow.<br />
<br />
As a bonus the last submitted selection is always saved as ''Last used''. This can be used when you want to make the same kml file again or make small changes without configuring everything again.<br />
<br />
It must be noted that the saved selections are browser specific. So will not show up when using a different computer.<br />
<br />
=== Limitations ===<br />
<br />
The KMZ generator will give a error when you try to make a KMZ file with more than 50.000 datapoints in total.<br />
The error will tell you how many data point are currently selected.<br />
<br />
===Visualization ===<br />
[[Image:GPS_visualization_screenshot1.png|thumb|right|560px|GPS track Visualization in Google Earth|]]<br />
<br />
The KMZ generator generates KMZ files containing one or more trackers. Currently, it saves the resulting file as t<trackers>-s<starttime>-e<endtime>.kmz , where <trackers> is a list of tracker identifiers separated by and ''_'', <starttime> is the start time in yyyymmddhhmm format and <endtime> is the endtime in yyyymmddhhmm format.<br />
The kmz itself can be viewed with a variety of applications, provided they support the KML format of files. In this documentation, we will limit ourselves to the use of Google Earth.<br />
<br />
==== Data Representation ====<br />
<br />
The data of each bird track consists of two different parts. The first part is a simple line of the chosen color showing the route that the bird has flown between the selected dates. The route is created by connecting all the GPS measurements in order of their timestamps. This gives a clear view of foraging and migratory habits over longer periods of time.<br />
<br />
The second part is a collection of icons, each one representing a GPS tag measurement on a certain time-step, and located on the latitude/longitude of the measurement. Each icon has several different visual indications of the measurement, such as:<br />
* The size of the icon is an indication of the altitude of the measurement, where the bigger icon represents a higher altitude.<br />
* The color of the icon is an indication of speed<br />
<br />
Because one bird track can consist of a large number of measurements, which are sometimes located in a small area.<br />
* Because each measurement has a timestamp attached to it, you can use the built in Google Earth time slider to reduce the number of measurements visible. <br />
<br />
=== Underlying Query and Data ===<br />
<br />
The following sql query is used:<br />
<br />
SELECT <br />
s.device_info_serial <br />
s.date_time, <br />
s.longitude, <br />
s.latitude, <br />
s.altitude, <br />
s.altitude_agl,<br />
round(CAST(s.speed_2d AS NUMERIC), 2) AS ispeed, <br />
round(CAST(ST_Length_Spheroid(ST_MakeLine(s.location, lag(s.location) OVER (ORDER BY s.device_info_serial, s.date_time)), 'SPHEROID["WGS 84",6378137,298.257223563]') / EXTRACT(epoch FROM s.date_time - lag(s.date_time) OVER (ORDER BY s.device_info_serial, s.date_time)) AS NUMERIC), 2) AS tspeed, <br />
round(s.direction, 2) AS idirection, <br />
round(CAST(degrees(ST_Azimuth(lag(s.location) OVER (ORDER BY s.device_info_serial, s.date_time), s.location)) AS NUMERIC), 2) AS tdirection,<br />
FROM gps.ee_tracking_speed_limited s<br />
WHERE <br />
s.device_info_serial = 583<br />
AND s.date_time BETWEEN '2013-04-01' AND '2013-06-01'<br />
AND s.longitude IS NOT NULL <br />
AND s.latitude IS NOT NULL <br />
AND s.userflag = 0<br />
ORDER BY s.date_time<br />
;</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=Tracker_Calendar&diff=1219Tracker Calendar2015-01-12T10:33:47Z<p>Stefan verhoeven: /* The UvA-BiTS Tracker calendar */</p>
<hr />
<div>== The UvA-BiTS Tracker calendar==<br />
<br />
The tracker calendar shows daily statistics or metrics of tracker as a calendar heatmap.<br />
It can be used to find days when something interesting happened or to find repeating patterns.<br />
<br />
The service itself can be found via the Virtual Lab [http://www.uva-bits.nl/virtual-lab/] by selecting "tracker calendar". This requires special access: You need to have credentials and your IP address should be registered. Please contact [mailto://UvA-BiTS@uva.nl UvA-BiTS@uva.nl] if you have problems getting access.<br />
<br />
=== Metrics ===<br />
<br />
The following metrics of a tracker are calculated for each day:<br />
<br />
* Nr. of GPS measurements<br />
* Nr. of accelerometer measurements<br />
* 2D distance travelled (km): Distance of line which consists of GPS measurements from midnight till midnight, altitude is ignored.<br />
* Maximum altitude (m): Highest absolute altitude<br />
* Average altitude (m): Average absolute altitude<br />
* Minimum altitude (m): Lowest absolute altitude<br />
* Maximum temperature (°C)<br />
* Average temperature (°C) <br />
* Minimum temperature (°C) <br />
* Minimum voltage battery (V): Lowest battery voltage<br />
* Maximum interval between GPS measurements (hh:mm:ss)<br />
* Minimum interval between GPS measurements (hh:mm:ss) <br />
<br />
=== System Requirements ===<br />
<br />
==== Software Requirements ====<br />
The webservice itself is standard javascript website, and as such is supported by all modern browsers and operating systems.<br />
<br />
=== The Submit Interface ===<br />
[[Image:CalendarSubmit.png|thumb|right|400px|abc|]]<br />
<br />
==== Select start date and end date ====<br />
<br />
For the begin and end date, there are two options: You can either type a date directly, or you can use the calender button. When you use the calender button, a small window pops up where a desired date can be selected. Here you can either use the arrows, or you can use ctrl up/down and ctrl left/right to change the years and months respectively. If you have selected a year and month, you can click on a day to finalize you date selection. For more detailed time tuning, there are also fields for the hours and minutes. Again, you can either type it, or you can use the drop down to select the desired hour.<br />
<br />
==== Select tracker ====<br />
<br />
To select tracker click on the row. The checkbox should be checked.<br />
<br />
==== Save selection ====<br />
<br />
Users often are interested in the same tracker. It would be tedious to select the same tracker each time. Therefore we added functionality to save and load specific selections.<br />
To save the current selection, press the ''Save selection'' button, choose a name, by default the current time will be used.<br />
To restore a saved selection, press the ''Restored saved selection'' button, double click on the name or press the icon with the arrow.<br />
<br />
As a bonus the last submitted selection is always saved as ''Last used''. This can be used when you want to make the same calendar again or make small changes without configuring everything again.<br />
<br />
It must be noted that the saved selections are browser specific. So will not show up when using a different computer.<br />
<br />
=== Limitations ===<br />
<br />
The Tracker calendar will give a error when you try to make a calendar with more than 1.000.000 datapoints.<br />
The error will tell you how many data point are currently selected.<br />
<br />
=== The Result Interface ===<br />
[[Image:CalendarResult.png|thumb|right|400px|abc|]]<br />
<br />
In the top left of the page you can see which tracker, date range and metric has been selected.<br />
In the top right of the page you can see the lowest and highest value for the currently selected tracker.<br />
The calendar is itself:<br />
<br />
* White background: no data for any metric on that day or day is outside selected date range<br />
* Gray background: no data for the selected metric on that day, but another metric has data<br />
* Moving mouse over a day will show the date and value of the metric<br />
<br />
On the bottom of the page you can download a csv file with the data shown in the calendar.<br />
<br />
==== Change metric ====<br />
<br />
The combobox at the top left corner of the page can be used to change to a different metric.<br />
<br />
==== Finding outliers ====<br />
<br />
The sliders in the top right corner of the page can be used to find days which are outliers.<br />
<br />
Dragging the left slider will change the minimum to a higher value, all days below this value will be colored 'blue'.<br />
<br />
Dragging the right slider will change the maximum to a lower value, all days above this value will be colored 'red'.<br />
<br />
Dragging the sliders will rescale the colors making it easier to see small changes.<br />
<br />
==== Change colors ====<br />
<br />
The default color range is lowest is light green and highest is dark green.<br />
The colors can be changed by <br />
<br />
# Press the 'Colors...' button in the top right corner of the page.<br />
# For the color you want to change press the '&or;' button<br />
# Pick a color<br />
# Press the 'Select Color' button</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=Flight_Generator&diff=1218Flight Generator2015-01-12T10:31:16Z<p>Stefan verhoeven: </p>
<hr />
<div>== Flight generator ==<br />
<br />
Doarama (http://http://www.doarama.com/) is a GPS track visualization tool. To add a track to Doarama a GPX file is required.<br />
A GPX file of a UvA-BiTS track can be generated using this service.<br />
<br />
The service itself can be found via the Virtual Lab [http://www.uva-bits.nl/virtual-lab/] by selecting "flight generator". This requires special access: You need to have credentials and your IP address should be registered. Please contact [mailto://UvA-BiTS@uva.nl UvA-BiTS@uva.nl] if you have problems getting access.<br />
<br />
To make nice flights you need high resolution data. http://www.doarama.com/view/135589 is an example of a foraging flight of our famous Lesser Black backed Gull that was breeding on Texel and went to Amsterdam for some time (http://www.uva-bits.nl/kml/visiting-amsterdam-for-a-day/) . This example shows nice high resolution data (16 sec interval; 9:00 -14:25), lower resolution data suggesting angulated flight (5 min interval; up to 8:20), and data that are difficult to interpret without landscape reference above the sea (11:00-15:00).<br />
<br />
=== System Requirements ===<br />
<br />
==== Software Requirements ====<br />
The webservice itself is standard javascript website, and as such is supported by all modern browsers and operating systems.<br />
<br />
==== Hardware requirements ====<br />
The Doarama (http://http://www.doarama.com/) visualization is a visual application, and therefore do require a little bit of graphical power. Fortunately, most modern computers have decent hardware for graphical acceleration, so this should not be a problem. But for example, if the graphical drivers are not installed correctly, or if remote desktop tools are used, performance can be limited. <br />
<br />
Additionally, Doarama require a decent internet connection. They use satellite images as background, and these have to be retrieved from the internet. Furthermore, if large amounts of data are selected in the flight generator, performance issues may arise.<br />
<br />
==== The interface ====<br />
<br />
=== The Submit Interface ===<br />
[[Image:GPXSubmit.png|thumb|right|400px|abc|]]<br />
<br />
==== Select start date and end date ====<br />
<br />
For the begin and end date, there are two options: You can either type a date directly, or you can use the calender button. When you use the calender button, a small window pops up where a desired date can be selected. Here you can either use the arrows, or you can use ctrl up/down and ctrl left/right to change the years and months respectively. If you have selected a year and month, you can click on a day to finalize you date selection. For more detailed time tuning, there are also fields for the hours and minutes. Again, you can either type it, or you can use the drop down to select the desired hour.<br />
<br />
==== Select tracker ====<br />
<br />
To select tracker click on the row. The checkbox should be checked.<br />
<br />
==== Save selection ====<br />
<br />
Users often are interested in the same tracker. It would be tedious to select the same tracker each time. Therefore we added functionality to save and load specific selections.<br />
To save the current selection, press the ''Save selection'' button, choose a name, by default the current time will be used.<br />
To restore a saved selection, press the ''Restored saved selection'' button, double click on the name or press the icon with the arrow.<br />
<br />
As a bonus the last submitted selection is always saved as ''Last used''. This can be used when you want to make the same gpx file again or make small changes without configuring everything again.<br />
<br />
It must be noted that the saved selections are browser specific. So will not show up when using a different computer.<br />
<br />
=== Limitations ===<br />
<br />
The Flight generator will give a error when you try to make a GPXfile with more than 50.000 datapoints.<br />
The error will tell you how many data point are currently selected.<br />
<br />
=== Visualization ===<br />
<br />
To make a Doarama visualization:<br />
<br />
# Download the GPX file from the result page<br />
# Goto http://www.doarama.com and Login<br />
# Click 'Upload' in topleft corner of page<br />
# Select the downloaded GPX file and upload it<br />
# Edit description <br />
# Set 'Activity Type' to 'Fly Bird / Aves'<br />
# Create a visualization<br />
# Press save button<br />
# The url that now appears in your browser is one you can share with others</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=Flight_Generator&diff=1217Flight Generator2015-01-12T10:28:47Z<p>Stefan verhoeven: </p>
<hr />
<div>== Flight generator ==<br />
<br />
Doarama (http://http://www.doarama.com/) is a GPS track visualization tool. To add a track to Doarama a GPX file is required.<br />
A GPX file of a UvA-BiTS track can be generated using this service.<br />
<br />
The service itself can be found via the Virtual Lab [http://www.uva-bits.nl/virtual-lab/] by selecting "flight generator". This requires special access: You need to have credentials and your IP address should be registered. Please contact [mailto://UvA-BiTS@uva.nl UvA-BiTS@uva.nl] if you have problems getting access.<br />
<br />
=== System Requirements ===<br />
<br />
==== Software Requirements ====<br />
The webservice itself is standard javascript website, and as such is supported by all modern browsers and operating systems.<br />
<br />
==== Hardware requirements ====<br />
The Doarama (http://http://www.doarama.com/) visualization is a visual application, and therefore do require a little bit of graphical power. Fortunately, most modern computers have decent hardware for graphical acceleration, so this should not be a problem. But for example, if the graphical drivers are not installed correctly, or if remote desktop tools are used, performance can be limited. <br />
<br />
Additionally, Doarama require a decent internet connection. They use satellite images as background, and these have to be retrieved from the internet. Furthermore, if large amounts of data are selected in the flight generator, performance issues may arise.<br />
<br />
==== The interface ====<br />
<br />
=== The Submit Interface ===<br />
[[Image:GPXSubmit.png|thumb|right|400px|abc|]]<br />
<br />
==== Select start date and end date ====<br />
<br />
For the begin and end date, there are two options: You can either type a date directly, or you can use the calender button. When you use the calender button, a small window pops up where a desired date can be selected. Here you can either use the arrows, or you can use ctrl up/down and ctrl left/right to change the years and months respectively. If you have selected a year and month, you can click on a day to finalize you date selection. For more detailed time tuning, there are also fields for the hours and minutes. Again, you can either type it, or you can use the drop down to select the desired hour.<br />
<br />
==== Select tracker ====<br />
<br />
To select tracker click on the row. The checkbox should be checked.<br />
<br />
==== Save selection ====<br />
<br />
Users often are interested in the same tracker. It would be tedious to select the same tracker each time. Therefore we added functionality to save and load specific selections.<br />
To save the current selection, press the ''Save selection'' button, choose a name, by default the current time will be used.<br />
To restore a saved selection, press the ''Restored saved selection'' button, double click on the name or press the icon with the arrow.<br />
<br />
As a bonus the last submitted selection is always saved as ''Last used''. This can be used when you want to make the same gpx file again or make small changes without configuring everything again.<br />
<br />
It must be noted that the saved selections are browser specific. So will not show up when using a different computer.<br />
<br />
=== Limitations ===<br />
<br />
The Flight generator will give a error when you try to make a GPXfile with more than 50.000 datapoints.<br />
The error will tell you how many data point are currently selected.<br />
<br />
=== Visualization ===<br />
<br />
To make a Doarama visualization:<br />
<br />
# Download the GPX file from the result page<br />
# Goto http://www.doarama.com and Login<br />
# Click 'Upload' in topleft corner of page<br />
# Select the downloaded GPX file and upload it<br />
# Edit description <br />
# Set 'Activity Type' to 'Fly Bird / Aves'<br />
# Create a visualization<br />
# Press save button</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=Flight_Generator&diff=1216Flight Generator2015-01-12T10:28:23Z<p>Stefan verhoeven: </p>
<hr />
<div>== Flight generator ==<br />
<br />
Doarama (http://http://www.doarama.com/) is a GPS track visualization tool. To add a track to Doarama a GPX file is required.<br />
A GPX file of a UvA-BiTS track can be generated using this service.<br />
<br />
The service itself can be found via the Virtual Lab [http://www.uva-bits.nl/virtual-lab/] by selecting "flight generator" or via direct link at [https://services.e-ecology.sara.nl/sw/tool/gpx/ e-ecology website]. This requires special access: You need to have credentials and your IP address should be registered. Please contact grid.support@sara.nl if you have problems getting access.<br />
<br />
=== System Requirements ===<br />
<br />
==== Software Requirements ====<br />
The webservice itself is standard javascript website, and as such is supported by all modern browsers and operating systems.<br />
<br />
==== Hardware requirements ====<br />
The Doarama (http://http://www.doarama.com/) visualization is a visual application, and therefore do require a little bit of graphical power. Fortunately, most modern computers have decent hardware for graphical acceleration, so this should not be a problem. But for example, if the graphical drivers are not installed correctly, or if remote desktop tools are used, performance can be limited. <br />
<br />
Additionally, Doarama require a decent internet connection. They use satellite images as background, and these have to be retrieved from the internet. Furthermore, if large amounts of data are selected in the flight generator, performance issues may arise.<br />
<br />
==== The interface ====<br />
<br />
=== The Submit Interface ===<br />
[[Image:GPXSubmit.png|thumb|right|400px|abc|]]<br />
<br />
==== Select start date and end date ====<br />
<br />
For the begin and end date, there are two options: You can either type a date directly, or you can use the calender button. When you use the calender button, a small window pops up where a desired date can be selected. Here you can either use the arrows, or you can use ctrl up/down and ctrl left/right to change the years and months respectively. If you have selected a year and month, you can click on a day to finalize you date selection. For more detailed time tuning, there are also fields for the hours and minutes. Again, you can either type it, or you can use the drop down to select the desired hour.<br />
<br />
==== Select tracker ====<br />
<br />
To select tracker click on the row. The checkbox should be checked.<br />
<br />
==== Save selection ====<br />
<br />
Users often are interested in the same tracker. It would be tedious to select the same tracker each time. Therefore we added functionality to save and load specific selections.<br />
To save the current selection, press the ''Save selection'' button, choose a name, by default the current time will be used.<br />
To restore a saved selection, press the ''Restored saved selection'' button, double click on the name or press the icon with the arrow.<br />
<br />
As a bonus the last submitted selection is always saved as ''Last used''. This can be used when you want to make the same gpx file again or make small changes without configuring everything again.<br />
<br />
It must be noted that the saved selections are browser specific. So will not show up when using a different computer.<br />
<br />
=== Limitations ===<br />
<br />
The Flight generator will give a error when you try to make a GPXfile with more than 50.000 datapoints.<br />
The error will tell you how many data point are currently selected.<br />
<br />
=== Visualization ===<br />
<br />
To make a Doarama visualization:<br />
<br />
# Download the GPX file from the result page<br />
# Goto http://www.doarama.com and Login<br />
# Click 'Upload' in topleft corner of page<br />
# Select the downloaded GPX file and upload it<br />
# Edit description <br />
# Set 'Activity Type' to 'Fly Bird / Aves'<br />
# Create a visualization<br />
# Press save button</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=Tracker_Calendar&diff=1215Tracker Calendar2015-01-12T10:27:37Z<p>Stefan verhoeven: /* Limitations */</p>
<hr />
<div>== The UvA-BiTS Tracker calendar==<br />
<br />
The UvA-BiTS GPS tracking data can aggregated for each day. This tracker calendar shows daily statistics or metrics of tracker as a calendar heatmap.<br />
It can be used to find days when something interesting happened or to find repeating patterns.<br />
<br />
The service itself can be found via the Virtual Lab [http://www.uva-bits.nl/virtual-lab/] by selecting "tracker calendar". This requires special access: You need to have credentials and your IP address should be registered. Please contact [mailto://UvA-BiTS@uva.nl UvA-BiTS@uva.nl] if you have problems getting access.<br />
<br />
=== Metrics ===<br />
<br />
The following metrics of a tracker are calculated for each day:<br />
<br />
* Nr. of GPS measurements<br />
* Nr. of accelerometer measurements<br />
* 2D distance travelled (km): Distance of line which consists of GPS measurements from midnight till midnight, altitude is ignored.<br />
* Maximum altitude (m): Highest absolute altitude<br />
* Average altitude (m): Average absolute altitude<br />
* Minimum altitude (m): Lowest absolute altitude<br />
* Maximum temperature (°C)<br />
* Average temperature (°C) <br />
* Minimum temperature (°C) <br />
* Minimum voltage battery (V): Lowest battery voltage<br />
* Maximum interval between GPS measurements (hh:mm:ss)<br />
* Minimum interval between GPS measurements (hh:mm:ss) <br />
<br />
=== System Requirements ===<br />
<br />
==== Software Requirements ====<br />
The webservice itself is standard javascript website, and as such is supported by all modern browsers and operating systems.<br />
<br />
=== The Submit Interface ===<br />
[[Image:CalendarSubmit.png|thumb|right|400px|abc|]]<br />
<br />
==== Select start date and end date ====<br />
<br />
For the begin and end date, there are two options: You can either type a date directly, or you can use the calender button. When you use the calender button, a small window pops up where a desired date can be selected. Here you can either use the arrows, or you can use ctrl up/down and ctrl left/right to change the years and months respectively. If you have selected a year and month, you can click on a day to finalize you date selection. For more detailed time tuning, there are also fields for the hours and minutes. Again, you can either type it, or you can use the drop down to select the desired hour.<br />
<br />
==== Select tracker ====<br />
<br />
To select tracker click on the row. The checkbox should be checked.<br />
<br />
==== Save selection ====<br />
<br />
Users often are interested in the same tracker. It would be tedious to select the same tracker each time. Therefore we added functionality to save and load specific selections.<br />
To save the current selection, press the ''Save selection'' button, choose a name, by default the current time will be used.<br />
To restore a saved selection, press the ''Restored saved selection'' button, double click on the name or press the icon with the arrow.<br />
<br />
As a bonus the last submitted selection is always saved as ''Last used''. This can be used when you want to make the same calendar again or make small changes without configuring everything again.<br />
<br />
It must be noted that the saved selections are browser specific. So will not show up when using a different computer.<br />
<br />
=== Limitations ===<br />
<br />
The Tracker calendar will give a error when you try to make a calendar with more than 1.000.000 datapoints.<br />
The error will tell you how many data point are currently selected.<br />
<br />
=== The Result Interface ===<br />
[[Image:CalendarResult.png|thumb|right|400px|abc|]]<br />
<br />
In the top left of the page you can see which tracker, date range and metirc has been selected.<br />
In the top right of the page you can see the lowest and highest value for the currently selected tracker.<br />
The calendar is itself:<br />
<br />
* White background: no data for any metric on that day or day is outside selected date range<br />
* Gray background: no data for the selected metric on that day, but another metric has data<br />
* Moving mouse over a day will show the date and value of the metric<br />
<br />
On the bottom of the page you can download a csv file with the data shown in the calendar.<br />
<br />
==== Change metric ====<br />
<br />
The combobox at the top left corner of the page can be used to change to a different metric.<br />
<br />
==== Finding outliers ====<br />
<br />
The sliders in the top right corner of the page can be used to find days which are outliers.<br />
<br />
Dragging the left slider will change the minimum to a higher value, all days below this value will be colored 'blue'.<br />
<br />
Dragging the right slider will change the maximum to a lower value, all days above this value will be colored 'red'.<br />
<br />
Dragging the sliders will rescale the colors making it easier to see small changes.<br />
<br />
==== Change colors ====<br />
<br />
The default color range is lowest is light green and highest is dark green.<br />
The colors can be changed by <br />
<br />
# Press the 'Colors...' button in the top right corner of the page.<br />
# For the color you want to change press the '&or;' button<br />
# Pick a color<br />
# Press the 'Select Color' button</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=Tracker_Calendar&diff=1214Tracker Calendar2015-01-12T10:27:17Z<p>Stefan verhoeven: /* The UvA-BiTS Tracker calendar */</p>
<hr />
<div>== The UvA-BiTS Tracker calendar==<br />
<br />
The UvA-BiTS GPS tracking data can aggregated for each day. This tracker calendar shows daily statistics or metrics of tracker as a calendar heatmap.<br />
It can be used to find days when something interesting happened or to find repeating patterns.<br />
<br />
The service itself can be found via the Virtual Lab [http://www.uva-bits.nl/virtual-lab/] by selecting "tracker calendar". This requires special access: You need to have credentials and your IP address should be registered. Please contact [mailto://UvA-BiTS@uva.nl UvA-BiTS@uva.nl] if you have problems getting access.<br />
<br />
=== Metrics ===<br />
<br />
The following metrics of a tracker are calculated for each day:<br />
<br />
* Nr. of GPS measurements<br />
* Nr. of accelerometer measurements<br />
* 2D distance travelled (km): Distance of line which consists of GPS measurements from midnight till midnight, altitude is ignored.<br />
* Maximum altitude (m): Highest absolute altitude<br />
* Average altitude (m): Average absolute altitude<br />
* Minimum altitude (m): Lowest absolute altitude<br />
* Maximum temperature (°C)<br />
* Average temperature (°C) <br />
* Minimum temperature (°C) <br />
* Minimum voltage battery (V): Lowest battery voltage<br />
* Maximum interval between GPS measurements (hh:mm:ss)<br />
* Minimum interval between GPS measurements (hh:mm:ss) <br />
<br />
=== System Requirements ===<br />
<br />
==== Software Requirements ====<br />
The webservice itself is standard javascript website, and as such is supported by all modern browsers and operating systems.<br />
<br />
=== The Submit Interface ===<br />
[[Image:CalendarSubmit.png|thumb|right|400px|abc|]]<br />
<br />
==== Select start date and end date ====<br />
<br />
For the begin and end date, there are two options: You can either type a date directly, or you can use the calender button. When you use the calender button, a small window pops up where a desired date can be selected. Here you can either use the arrows, or you can use ctrl up/down and ctrl left/right to change the years and months respectively. If you have selected a year and month, you can click on a day to finalize you date selection. For more detailed time tuning, there are also fields for the hours and minutes. Again, you can either type it, or you can use the drop down to select the desired hour.<br />
<br />
==== Select tracker ====<br />
<br />
To select tracker click on the row. The checkbox should be checked.<br />
<br />
==== Save selection ====<br />
<br />
Users often are interested in the same tracker. It would be tedious to select the same tracker each time. Therefore we added functionality to save and load specific selections.<br />
To save the current selection, press the ''Save selection'' button, choose a name, by default the current time will be used.<br />
To restore a saved selection, press the ''Restored saved selection'' button, double click on the name or press the icon with the arrow.<br />
<br />
As a bonus the last submitted selection is always saved as ''Last used''. This can be used when you want to make the same calendar again or make small changes without configuring everything again.<br />
<br />
It must be noted that the saved selections are browser specific. So will not show up when using a different computer.<br />
<br />
=== Limitations ===<br />
<br />
The Tracker calendar will give a error when you try to make a calendar with more than 1.000.000 datapoints in total.<br />
The error will tell you how many data point are currently selected.<br />
<br />
=== The Result Interface ===<br />
[[Image:CalendarResult.png|thumb|right|400px|abc|]]<br />
<br />
In the top left of the page you can see which tracker, date range and metirc has been selected.<br />
In the top right of the page you can see the lowest and highest value for the currently selected tracker.<br />
The calendar is itself:<br />
<br />
* White background: no data for any metric on that day or day is outside selected date range<br />
* Gray background: no data for the selected metric on that day, but another metric has data<br />
* Moving mouse over a day will show the date and value of the metric<br />
<br />
On the bottom of the page you can download a csv file with the data shown in the calendar.<br />
<br />
==== Change metric ====<br />
<br />
The combobox at the top left corner of the page can be used to change to a different metric.<br />
<br />
==== Finding outliers ====<br />
<br />
The sliders in the top right corner of the page can be used to find days which are outliers.<br />
<br />
Dragging the left slider will change the minimum to a higher value, all days below this value will be colored 'blue'.<br />
<br />
Dragging the right slider will change the maximum to a lower value, all days above this value will be colored 'red'.<br />
<br />
Dragging the sliders will rescale the colors making it easier to see small changes.<br />
<br />
==== Change colors ====<br />
<br />
The default color range is lowest is light green and highest is dark green.<br />
The colors can be changed by <br />
<br />
# Press the 'Colors...' button in the top right corner of the page.<br />
# For the color you want to change press the '&or;' button<br />
# Pick a color<br />
# Press the 'Select Color' button</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=KMZ_Generator2&diff=1213KMZ Generator22015-01-12T10:25:08Z<p>Stefan verhoeven: /* The UvA-BiTS KMZ Generator */ Added limit</p>
<hr />
<div>== The UvA-BiTS KMZ Generator==<br />
<br />
The UvA-BiTS GPS tracking data can be overlaid on maps in a broad range of software packages. This track generator creates a [http://en.wikipedia.org/wiki/Keyhole_Markup_Language KMZ] file which can be loaded into Google Earth, as well as many other GIS-systems which provide KMZ reading capabilities.<br />
<br />
The service itself can be found via the Virtual Lab [http://www.uva-bits.nl/virtual-lab/] by selecting "kmz generator". This requires special access: You need to have credentials and your IP address should be registered. Please contact [mailto://UvA-BiTS@uva.nl UvA-BiTS@uva.nl] if you have problems getting access.<br />
<br />
=== System Requirements ===<br />
<br />
==== Software Requirements ====<br />
The webservice itself is standard javascript website, and as such is supported by all modern browsers and operating systems.<br />
<br />
For looking at KML files, there are multiple options. The options we support are [http://earth.google.com/ Google Earth] and [http://public.flysafe.sara.nl/birdview.html Birdview].<br />
<br />
==== Hardware requirements ====<br />
Both [http://earth.google.com/ Google Earth] and [http://birdvis.e-ecology.sara.nl/birdview.html Birdview] are visual applications, and therefore do require a little bit of graphical power. Fortunately, most modern computers have decent hardware for graphical acceleration, so this should not be a problem. But for example, if the graphical drivers are not installed correctly, or if remote desktop tools are used, performance can be limited. <br />
<br />
Additionally, both [http://earth.google.com/ Google Earth] and [http://birdvis.e-ecology.sara.nl/birdview.html Birdview] require a decent internet connection. They use satellite images as background, and these have to be retrieved from the internet. Furthermore, if large amounts of data are selected in the kmz generator, performance issues may arise.<br />
<br />
=== The Interface ===<br />
[[Image:Swkmzgen.png|thumb|right|400px|abc|]]<br />
<br />
Records exist from May 2008 and onwards.<br />
In order to be able to select specific bird(s) and timespan(s) for detailed studying, an interface was built that allows users to query and retrieve specified subsets of the available data. See the image to the right for a screenshot of the interface.<br />
<br />
To use the KMZ generator:<br />
# Select start date and end date<br />
# Select trackers<br />
# Customize tracker color<br />
# Advanced settings (optionally)<br />
# Save selection (optionally)<br />
# Press submit button<br />
# Wait for kmz to be generated<br />
# Download kmz and open it Google Earth<br />
<br />
==== Select start date and end date ====<br />
<br />
For the begin and end date, there are two options: You can either type a date directly, or you can use the calendar button. When you use the calendar button, a small window pops up where a desired date can be selected. Here you can either use the arrows, or you can use ctrl up/down and ctrl left/right to change the years and months respectively. If you have selected a year and month, you can click on a day to finalize you date selection. For more detailed time tuning, there are also fields for the hours and minutes. Again, you can either type it, or you can use the drop down to select the desired hour.<br />
<br />
==== Select trackers ====<br />
<br />
To select trackers they must be moved from the 'Available' table to the 'Selected' table. Moving tracker can be done in 3 ways:<br />
* double click on tracker<br />
* drag and drop<br />
* select it and press arrow button<br />
Multiple trackers can be selected at the same time.<br />
<br />
==== Customize tracker color ====<br />
<br />
The selected trackers are automatically assigned colors. The colors can be changed by clicking on the colored bar or pencil icon. A list of colors will be shown, clicking will select it.<br />
<br />
==== Advanced settings ====<br />
<br />
The advanced settings can be opened by clicking on the ''Advanced settings'' text on between the tracker selection and submit buttons.<br />
Available settings:<br />
* Shape, use a circle icon or an directional arrow. The direction can be based on instantaneous direction (the direction the GPS tracker reported) or traject direction (direction between current point and previous point)<br />
* Size, use small, medium or large icons<br />
* Size based on altitude, when checked the icons will be bigger when altitude is higher.<br />
* Color, can be single color or a gradient. Which color gradiention to use is based on the speed. The higher the speed the darker. Same as directional arrow there is a choice between instantaneous and trajectory speed.<br />
* Speed thresholds, when speed gradient is selected for color then you can choose at which speed which color should be used.<br />
* Transparency, when you have many points or lines in the same place transparency can be used to make density map by making the points and lines more transparent.<br />
* Altitude, the altitude of points can be configure to be: <br />
** absolute, will use the altitude of the tracker<br />
** clamp to ground, will snap the point to the ground<br />
** relative to ground, the ground elevation will be subtracted from the altitude of the tracker<br />
<br />
==== Save selection ====<br />
<br />
Users often are interested in the same trackers. It would be tedious to select the same trackers each time. Therefore we added functionality to save and load specific selections.<br />
To save the current selection, press the ''Save selection'' button, choose a name, by default the current time will be used.<br />
To restore a saved selection, press the ''Restored saved selection'' button, double click on the name or press the icon with the arrow.<br />
<br />
As a bonus the last submitted selection is always saved as ''Last used''. This can be used when you want to make the same kml file again or make small changes without configuring everything again.<br />
<br />
It must be noted that the saved selections are browser specific. So will not show up when using a different computer.<br />
<br />
=== Limitations ===<br />
<br />
The KMZ generator will give a error when you try to make a KMZ file with more than 50.000 datapoints in total.<br />
The error will tell you how many data point are currently selected.<br />
<br />
===Visualization ===<br />
[[Image:GPS_visualization_screenshot1.png|thumb|right|560px|GPS track Visualization in Google Earth|]]<br />
<br />
The KMZ generator generates KMZ files containing one or more trackers. Currently, it saves the resulting file as t<trackers>-s<starttime>-e<endtime>.kmz , where <trackers> is a list of tracker identifiers separated by and ''_'', <starttime> is the start time in yyyymmddhhmm format and <endtime> is the endtime in yyyymmddhhmm format.<br />
The kmz itself can be viewed with a variety of applications, provided they support the KML format of files. In this documentation, we will limit ourselves to the use of Google Earth.<br />
<br />
==== Data Representation ====<br />
<br />
The data of each bird track consists of two different parts. The first part is a simple line of the chosen color showing the route that the bird has flown between the selected dates. The route is created by connecting all the GPS measurements in order of their timestamps. This gives a clear view of foraging and migratory habits over longer periods of time.<br />
<br />
The second part is a collection of icons, each one representing a GPS tag measurement on a certain time-step, and located on the latitude/longitude of the measurement. Each icon has several different visual indications of the measurement, such as:<br />
* The size of the icon is an indication of the altitude of the measurement, where the bigger icon represents a higher altitude.<br />
* The color of the icon is an indication of speed<br />
<br />
Because one bird track can consist of a large number of measurements, which are sometimes located in a small area.<br />
* Because each measurement has a timestamp attached to it, you can use the built in Google Earth time slider to reduce the number of measurements visible. <br />
<br />
=== Underlying Query and Data ===<br />
<br />
The following sql query is used:<br />
<br />
SELECT <br />
s.device_info_serial <br />
s.date_time, <br />
s.longitude, <br />
s.latitude, <br />
s.altitude, <br />
s.altitude_agl,<br />
round(CAST(s.speed_2d AS NUMERIC), 2) AS ispeed, <br />
round(CAST(ST_Length_Spheroid(ST_MakeLine(s.location, lag(s.location) OVER (ORDER BY s.device_info_serial, s.date_time)), 'SPHEROID["WGS 84",6378137,298.257223563]') / EXTRACT(epoch FROM s.date_time - lag(s.date_time) OVER (ORDER BY s.device_info_serial, s.date_time)) AS NUMERIC), 2) AS tspeed, <br />
round(s.direction, 2) AS idirection, <br />
round(CAST(degrees(ST_Azimuth(lag(s.location) OVER (ORDER BY s.device_info_serial, s.date_time), s.location)) AS NUMERIC), 2) AS tdirection,<br />
FROM gps.ee_tracking_speed_limited s<br />
WHERE <br />
s.device_info_serial = 583<br />
AND s.date_time BETWEEN '2013-04-01' AND '2013-06-01'<br />
AND s.longitude IS NOT NULL <br />
AND s.latitude IS NOT NULL <br />
AND s.userflag = 0<br />
ORDER BY s.date_time<br />
;</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=KMZ_Generator2&diff=1212KMZ Generator22015-01-12T10:18:36Z<p>Stefan verhoeven: Merged suggestions of Willem</p>
<hr />
<div>== The UvA-BiTS KMZ Generator==<br />
<br />
The UvA-BiTS GPS tracking data can be overlaid on maps in a broad range of software packages. This track generator creates a [http://en.wikipedia.org/wiki/Keyhole_Markup_Language KMZ] file which can be loaded into Google Earth, as well as many other GIS-systems which provide KMZ reading capabilities.<br />
<br />
The service itself can be found via the Virtual Lab [http://www.uva-bits.nl/virtual-lab/] by selecting "kmz generator". This requires special access: You need to have credentials and your IP address should be registered. Please contact UvA-BiTS@uva.nl if you have problems getting access.<br />
<br />
=== System Requirements ===<br />
<br />
==== Software Requirements ====<br />
The webservice itself is standard javascript website, and as such is supported by all modern browsers and operating systems.<br />
<br />
For looking at KML files, there are multiple options. The options we support are [http://earth.google.com/ Google Earth] and [http://public.flysafe.sara.nl/birdview.html Birdview].<br />
<br />
==== Hardware requirements ====<br />
Both [http://earth.google.com/ Google Earth] and [http://birdvis.e-ecology.sara.nl/birdview.html Birdview] are visual applications, and therefore do require a little bit of graphical power. Fortunately, most modern computers have decent hardware for graphical acceleration, so this should not be a problem. But for example, if the graphical drivers are not installed correctly, or if remote desktop tools are used, performance can be limited. <br />
<br />
Additionally, both [http://earth.google.com/ Google Earth] and [http://birdvis.e-ecology.sara.nl/birdview.html Birdview] require a decent internet connection. They use satellite images as background, and these have to be retrieved from the internet. Furthermore, if large amounts of data are selected in the kmz generator, performance issues may arise.<br />
<br />
=== The Interface ===<br />
[[Image:Swkmzgen.png|thumb|right|400px|abc|]]<br />
<br />
Records exist from May 2008 and onwards.<br />
In order to be able to select specific bird(s) and timespan(s) for detailed studying, an interface was built that allows users to query and retrieve specified subsets of the available data. See the image to the right for a screenshot of the interface.<br />
<br />
To use the KMZ generator:<br />
# Select start date and end date<br />
# Select trackers<br />
# Customize tracker color<br />
# Advanced settings (optionally)<br />
# Save selection (optionally)<br />
# Press submit button<br />
# Wait for kmz to be generated<br />
# Download kmz and open it Google Earth<br />
<br />
==== Select start date and end date ====<br />
<br />
For the begin and end date, there are two options: You can either type a date directly, or you can use the calendar button. When you use the calendar button, a small window pops up where a desired date can be selected. Here you can either use the arrows, or you can use ctrl up/down and ctrl left/right to change the years and months respectively. If you have selected a year and month, you can click on a day to finalize you date selection. For more detailed time tuning, there are also fields for the hours and minutes. Again, you can either type it, or you can use the drop down to select the desired hour.<br />
<br />
==== Select trackers ====<br />
<br />
To select trackers they must be moved from the 'Available' table to the 'Selected' table. Moving tracker can be done in 3 ways:<br />
* double click on tracker<br />
* drag and drop<br />
* select it and press arrow button<br />
Multiple trackers can be selected at the same time.<br />
<br />
==== Customize tracker color ====<br />
<br />
The selected trackers are automatically assigned colors. The colors can be changed by clicking on the colored bar or pencil icon. A list of colors will be shown, clicking will select it.<br />
<br />
==== Advanced settings ====<br />
<br />
The advanced settings can be opened by clicking on the ''Advanced settings'' text on between the tracker selection and submit buttons.<br />
Available settings:<br />
* Shape, use a circle icon or an directional arrow. The direction can be based on instantaneous direction (the direction the GPS tracker reported) or traject direction (direction between current point and previous point)<br />
* Size, use small, medium or large icons<br />
* Size based on altitude, when checked the icons will be bigger when altitude is higher.<br />
* Color, can be single color or a gradient. Which color gradiention to use is based on the speed. The higher the speed the darker. Same as directional arrow there is a choice between instantaneous and trajectory speed.<br />
* Speed thresholds, when speed gradient is selected for color then you can choose at which speed which color should be used.<br />
* Transparency, when you have many points or lines in the same place transparency can be used to make density map by making the points and lines more transparent.<br />
* Altitude, the altitude of points can be configure to be: <br />
** absolute, will use the altitude of the tracker<br />
** clamp to ground, will snap the point to the ground<br />
** relative to ground, the ground elevation will be subtracted from the altitude of the tracker<br />
<br />
==== Save selection ====<br />
<br />
Users often are interested in the same trackers. It would be tedious to select the same trackers each time. Therefore we added functionality to save and load specific selections.<br />
To save the current selection, press the ''Save selection'' button, choose a name, by default the current time will be used.<br />
To restore a saved selection, press the ''Restored saved selection'' button, double click on the name or press the icon with the arrow.<br />
<br />
As a bonus the last submitted selection is always saved as ''Last used''. This can be used when you want to make the same kml file again or make small changes without configuring everything again.<br />
<br />
It must be noted that the saved selections are browser specific. So will not show up when using a different computer.<br />
<br />
===Visualization ===<br />
[[Image:GPS_visualization_screenshot1.png|thumb|right|560px|GPS track Visualization in Google Earth|]]<br />
<br />
The KMZ generator generates KMZ files containing one or more trackers. Currently, it saves the resulting file as t<trackers>-s<starttime>-e<endtime>.kmz , where <trackers> is a list of tracker identifiers separated by and ''_'', <starttime> is the start time in yyyymmddhhmm format and <endtime> is the endtime in yyyymmddhhmm format.<br />
The kmz itself can be viewed with a variety of applications, provided they support the KML format of files. In this documentation, we will limit ourselves to the use of Google Earth.<br />
<br />
==== Data Representation ====<br />
<br />
The data of each bird track consists of two different parts. The first part is a simple line of the chosen color showing the route that the bird has flown between the selected dates. The route is created by connecting all the GPS measurements in order of their timestamps. This gives a clear view of foraging and migratory habits over longer periods of time.<br />
<br />
The second part is a collection of icons, each one representing a GPS tag measurement on a certain time-step, and located on the latitude/longitude of the measurement. Each icon has several different visual indications of the measurement, such as:<br />
* The size of the icon is an indication of the altitude of the measurement, where the bigger icon represents a higher altitude.<br />
* The color of the icon is an indication of speed<br />
<br />
Because one bird track can consist of a large number of measurements, which are sometimes located in a small area.<br />
* Because each measurement has a timestamp attached to it, you can use the built in Google Earth time slider to reduce the number of measurements visible. <br />
<br />
=== Underlying Query and Data ===<br />
<br />
The following sql query is used:<br />
<br />
SELECT <br />
s.device_info_serial <br />
s.date_time, <br />
s.longitude, <br />
s.latitude, <br />
s.altitude, <br />
s.altitude_agl,<br />
round(CAST(s.speed_2d AS NUMERIC), 2) AS ispeed, <br />
round(CAST(ST_Length_Spheroid(ST_MakeLine(s.location, lag(s.location) OVER (ORDER BY s.device_info_serial, s.date_time)), 'SPHEROID["WGS 84",6378137,298.257223563]') / EXTRACT(epoch FROM s.date_time - lag(s.date_time) OVER (ORDER BY s.device_info_serial, s.date_time)) AS NUMERIC), 2) AS tspeed, <br />
round(s.direction, 2) AS idirection, <br />
round(CAST(degrees(ST_Azimuth(lag(s.location) OVER (ORDER BY s.device_info_serial, s.date_time), s.location)) AS NUMERIC), 2) AS tdirection,<br />
FROM gps.ee_tracking_speed_limited s<br />
WHERE <br />
s.device_info_serial = 583<br />
AND s.date_time BETWEEN '2013-04-01' AND '2013-06-01'<br />
AND s.longitude IS NOT NULL <br />
AND s.latitude IS NOT NULL <br />
AND s.userflag = 0<br />
ORDER BY s.date_time<br />
;</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=E-Ecology_Documentation_and_Guidance&diff=1211E-Ecology Documentation and Guidance2015-01-12T09:51:52Z<p>Stefan verhoeven: </p>
<hr />
<div>The e-Ecology Virtual Laboratories represent a collaborative effort between the University of Amsterdam Institute for Biodiversity and Ecosystem Dynamics and SARA Computing to manage bird tracking data and develop tools for data exploration, analysis and visualization. This wiki is designed to facilitate access to the data and tools within the Virtual Laboratories and to provide a forum for collaboration. <br />
<br />
== UvA-BiTS data == <br />
<br />
:Bird data collected with the UvA-BiTS Bird Tracking System<br />
<br />
Note: We are continually working on improving UvA-BiTS services, including data processing and Virtual Lab functionality. In some cases, this has enabled us to extract more information from old GPS log files which is then incorporated into the database. These changes may result in slight differences when you reprocess historic data. <br />
<br />
::* [[ DB_Model_2014 | e-Ecology data model]]<br />
::* [[ UvAGPS|UvA-BiTS Data Model]] - OLD<br />
::* [[Connecting to the UvA-BiTS database]]<br />
::* [[Useful Queries]]<br />
::* [[Queries for linking bird tracks to wind and flow data]]<br />
<br />
==UvA-BiTS Services==<br />
<br />
* Visualization Tools<br />
:: [[KMZ Generator2]]: <br /><br />
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth. <br />
:: [[Tracker Calendar]]: <br /><br />
:::Calendar overview with daily statistics of GPS-tracker<br />
:: [[Flight Generator]]: <br /><br />
:::This tool produces GPX files from UvA-BiTS tracking data for dynamic [http://www.doarama.com Doarama] visualizations.<br />
:: [[BirdView]]: <br /><br />
:::Access meterological and radar data and overlay with GPS bird tracking data.<br />
:: [[BirdSim]]: <br/ ><br />
:::More dynamic version of BirdView that allows the user to scroll through weather data in time and space.<br />
:: [[KMZ Generator|UvA-BiTS KMZ Generator]]: <br /><br />
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth. <br />
<br />
* [[Dashboard]]: These web pages allow UvA-BiTS users to get an overview of the raw data files of GPS bird tracking devices.<br />
<br />
* [[ Annotation tables ]] for uploading GPS annotation data into your own schema. This service is currently not available.<br />
<br />
== Changes in the Database in 2014 ==<br />
'''Important'''<br />
In the summer of 2014 we have '''re-arranged the administrative''' part of UvA-BiTS (and e-Ecology) in the database. End users may have to adapt their programs the following way:<br />
:* The '''uva_*_limited''' views that used to exist in the gps schema are gone. Most of them should have been replaced with an equivalent '''ee_*_limited counterpart'''. <br />
:* The new ee_*_limited views show a superset of the same information as before, but enhanced with new project information.<br />
:* Some '''column names''' in the ee_*_limited views may differ from the previous views.<br />
:* There are new limited views: <br />
:** gps.ee_'''project'''_limited: shows your projects' information<br />
:** gps.ee_'''shared'''_*_limited: set of data you are able to see related to trackers that a project is sharing with a project you belong to<br />
:* Special attention: gps.uva_device_limited is now replaced by '''gps.ee_tracker_limited'''. For each tracker, you now see which project provides you access to that tracker.<br />
:* Also, '''gps.uva_tracking_limited''' is gone. Instead of that we have gps.ee_tracker_speed_limited, which adds speed columns to the original data. This amounts to a merger of gps.ee_tracking_limited, gps.ee_tracking_speed_3d_limited and gps.ee_tracking_speed_limited.<br />
<br />
==Other e-Ecology Datasets==<br />
<br />
'''ENVIRONMENTAL DATA'''<br />
<br />
::* [[ ECMWF|Weather Forecast Data (ECMWF)]]<br />
::* [[ Elevation|Elevation Data ]]<br />
<br />
'''FLYSAFE'''<br />
<br />
:Bird data created from weather radar data and bird migration models. Financed by the Royal Netherlands Air Force. <br />
<br />
::* [[Connecting to the Flysafe database]]<br />
::* [[GPS|Flysafe GPS Data Model]]<br />
::* [[ RADAR|Radar Data Model]]<br />
::* [[ Wind profile|Wind Profile Data Model]]<br />
::* [[ Fino1|Fino1 Data Model]]<br />
<br />
==Other e-Ecology Services==<br />
[[GPS tracks|FlySafe GPS Bird Track Generator]]: Produce kmz files of ARGOS/GPS bird tracking data from the FlySafe project to be viewed in Google Earth. <br />
<br />
[[ Administration]]: Information about user rights and data processing tasks.<br />
<br />
[[ Bookkeeping]]: Description of tables related to FlySafe data processing and archiving.<br />
<br />
[[ ToDo ]]</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=Tracker_Calendar&diff=1209Tracker Calendar2015-01-12T09:51:41Z<p>Stefan verhoeven: moved Calendar to Tracker Calendar</p>
<hr />
<div>== The UvA-BiTS Tracker calendar==<br />
<br />
The UvA-BiTS GPS tracking data can aggregated for each day. This tracker calendar shows daily statistics or metrics of tracker as a calendar heatmap.<br />
It can be used to find days when something interesting happened or to find repeating patterns.<br />
<br />
The service itself can be found via the Virtual Lab [http://www.uva-bits.nl/virtual-lab/] by selecting "tracker calendar" or via direct link at [https://services.e-ecology.sara.nl/sw/tool/calendar/ e-ecology website]. This requires special access: You need to have credentials and your IP address should be registered. Please contact grid.support@sara.nl if you have problems getting access.<br />
<br />
=== Metrics ===<br />
<br />
The following metrics of a tracker are calculated for each day:<br />
<br />
* Nr. of GPS measurements<br />
* Nr. of accelerometer measurements<br />
* 2D distance travelled (km): Distance of line which consists of GPS measurements from midnight till midnight, altitude is ignored.<br />
* Maximum altitude (m): Highest absolute altitude<br />
* Average altitude (m): Average absolute altitude<br />
* Minimum altitude (m): Lowest absolute altitude<br />
* Maximum temperature (°C)<br />
* Average temperature (°C) <br />
* Minimum temperature (°C) <br />
* Minimum voltage battery (V): Lowest battery voltage<br />
* Maximum interval between GPS measurements (hh:mm:ss)<br />
* Minimum interval between GPS measurements (hh:mm:ss) <br />
<br />
=== System Requirements ===<br />
<br />
==== Software Requirements ====<br />
The webservice itself is standard javascript website, and as such is supported by all modern browsers and operating systems.<br />
<br />
=== The Submit Interface ===<br />
[[Image:CalendarSubmit.png|thumb|right|400px|abc|]]<br />
<br />
==== Select start date and end date ====<br />
<br />
For the begin and end date, there are two options: You can either type a date directly, or you can use the calender button. When you use the calender button, a small window pops up where a desired date can be selected. Here you can either use the arrows, or you can use ctrl up/down and ctrl left/right to change the years and months respectively. If you have selected a year and month, you can click on a day to finalize you date selection. For more detailed time tuning, there are also fields for the hours and minutes. Again, you can either type it, or you can use the drop down to select the desired hour.<br />
<br />
==== Select tracker ====<br />
<br />
To select tracker click on the row. The checkbox should be checked.<br />
<br />
==== Save selection ====<br />
<br />
Users often are interested in the same tracker. It would be tedious to select the same tracker each time. Therefore we added functionality to save and load specific selections.<br />
To save the current selection, press the ''Save selection'' button, choose a name, by default the current time will be used.<br />
To restore a saved selection, press the ''Restored saved selection'' button, double click on the name or press the icon with the arrow.<br />
<br />
As a bonus the last submitted selection is always saved as ''Last used''. This can be used when you want to make the same calendar again or make small changes without configuring everything again.<br />
<br />
It must be noted that the saved selections are browser specific. So will not show up when using a different computer.<br />
<br />
=== The Result Interface ===<br />
[[Image:CalendarResult.png|thumb|right|400px|abc|]]<br />
<br />
In the top left of the page you can see which tracker, date range and metirc has been selected.<br />
In the top right of the page you can see the lowest and highest value for the currently selected tracker.<br />
The calendar is itself:<br />
<br />
* White background: no data for any metric on that day or day is outside selected date range<br />
* Gray background: no data for the selected metric on that day, but another metric has data<br />
* Moving mouse over a day will show the date and value of the metric<br />
<br />
On the bottom of the page you can download a csv file with the data shown in the calendar.<br />
<br />
==== Change metric ====<br />
<br />
The combobox at the top left corner of the page can be used to change to a different metric.<br />
<br />
==== Finding outliers ====<br />
<br />
The sliders in the top right corner of the page can be used to find days which are outliers.<br />
<br />
Dragging the left slider will change the minimum to a higher value, all days below this value will be colored 'blue'.<br />
<br />
Dragging the right slider will change the maximum to a lower value, all days above this value will be colored 'red'.<br />
<br />
Dragging the sliders will rescale the colors making it easier to see small changes.<br />
<br />
==== Change colors ====<br />
<br />
The default color range is lowest is light green and highest is dark green.<br />
The colors can be changed by <br />
<br />
# Press the 'Colors...' button in the top right corner of the page.<br />
# For the color you want to change press the '&or;' button<br />
# Pick a color<br />
# Press the 'Select Color' button</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=Calendar&diff=1210Calendar2015-01-12T09:51:41Z<p>Stefan verhoeven: moved Calendar to Tracker Calendar</p>
<hr />
<div>#REDIRECT [[Tracker Calendar]]</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=E-Ecology_Documentation_and_Guidance&diff=1208E-Ecology Documentation and Guidance2015-01-12T09:51:20Z<p>Stefan verhoeven: </p>
<hr />
<div>The e-Ecology Virtual Laboratories represent a collaborative effort between the University of Amsterdam Institute for Biodiversity and Ecosystem Dynamics and SARA Computing to manage bird tracking data and develop tools for data exploration, analysis and visualization. This wiki is designed to facilitate access to the data and tools within the Virtual Laboratories and to provide a forum for collaboration. <br />
<br />
== UvA-BiTS data == <br />
<br />
:Bird data collected with the UvA-BiTS Bird Tracking System<br />
<br />
Note: We are continually working on improving UvA-BiTS services, including data processing and Virtual Lab functionality. In some cases, this has enabled us to extract more information from old GPS log files which is then incorporated into the database. These changes may result in slight differences when you reprocess historic data. <br />
<br />
::* [[ DB_Model_2014 | e-Ecology data model]]<br />
::* [[ UvAGPS|UvA-BiTS Data Model]] - OLD<br />
::* [[Connecting to the UvA-BiTS database]]<br />
::* [[Useful Queries]]<br />
::* [[Queries for linking bird tracks to wind and flow data]]<br />
<br />
==UvA-BiTS Services==<br />
<br />
* Visualization Tools<br />
:: [[KMZ Generator2]]: <br /><br />
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth. <br />
:: [[Calendar]]: <br /><br />
:::Calendar overview with daily statistics of GPS-tracker<br />
:: [[Flight Generator]]: <br /><br />
:::This tool produces GPX files from UvA-BiTS tracking data for dynamic [http://www.doarama.com Doarama] visualizations.<br />
:: [[BirdView]]: <br /><br />
:::Access meterological and radar data and overlay with GPS bird tracking data.<br />
:: [[BirdSim]]: <br/ ><br />
:::More dynamic version of BirdView that allows the user to scroll through weather data in time and space.<br />
:: [[KMZ Generator|UvA-BiTS KMZ Generator]]: <br /><br />
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth. <br />
<br />
* [[Dashboard]]: These web pages allow UvA-BiTS users to get an overview of the raw data files of GPS bird tracking devices.<br />
<br />
* [[ Annotation tables ]] for uploading GPS annotation data into your own schema. This service is currently not available.<br />
<br />
== Changes in the Database in 2014 ==<br />
'''Important'''<br />
In the summer of 2014 we have '''re-arranged the administrative''' part of UvA-BiTS (and e-Ecology) in the database. End users may have to adapt their programs the following way:<br />
:* The '''uva_*_limited''' views that used to exist in the gps schema are gone. Most of them should have been replaced with an equivalent '''ee_*_limited counterpart'''. <br />
:* The new ee_*_limited views show a superset of the same information as before, but enhanced with new project information.<br />
:* Some '''column names''' in the ee_*_limited views may differ from the previous views.<br />
:* There are new limited views: <br />
:** gps.ee_'''project'''_limited: shows your projects' information<br />
:** gps.ee_'''shared'''_*_limited: set of data you are able to see related to trackers that a project is sharing with a project you belong to<br />
:* Special attention: gps.uva_device_limited is now replaced by '''gps.ee_tracker_limited'''. For each tracker, you now see which project provides you access to that tracker.<br />
:* Also, '''gps.uva_tracking_limited''' is gone. Instead of that we have gps.ee_tracker_speed_limited, which adds speed columns to the original data. This amounts to a merger of gps.ee_tracking_limited, gps.ee_tracking_speed_3d_limited and gps.ee_tracking_speed_limited.<br />
<br />
==Other e-Ecology Datasets==<br />
<br />
'''ENVIRONMENTAL DATA'''<br />
<br />
::* [[ ECMWF|Weather Forecast Data (ECMWF)]]<br />
::* [[ Elevation|Elevation Data ]]<br />
<br />
'''FLYSAFE'''<br />
<br />
:Bird data created from weather radar data and bird migration models. Financed by the Royal Netherlands Air Force. <br />
<br />
::* [[Connecting to the Flysafe database]]<br />
::* [[GPS|Flysafe GPS Data Model]]<br />
::* [[ RADAR|Radar Data Model]]<br />
::* [[ Wind profile|Wind Profile Data Model]]<br />
::* [[ Fino1|Fino1 Data Model]]<br />
<br />
==Other e-Ecology Services==<br />
[[GPS tracks|FlySafe GPS Bird Track Generator]]: Produce kmz files of ARGOS/GPS bird tracking data from the FlySafe project to be viewed in Google Earth. <br />
<br />
[[ Administration]]: Information about user rights and data processing tasks.<br />
<br />
[[ Bookkeeping]]: Description of tables related to FlySafe data processing and archiving.<br />
<br />
[[ ToDo ]]</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=E-Ecology_Documentation_and_Guidance&diff=1207E-Ecology Documentation and Guidance2015-01-12T09:51:07Z<p>Stefan verhoeven: </p>
<hr />
<div>The e-Ecology Virtual Laboratories represent a collaborative effort between the University of Amsterdam Institute for Biodiversity and Ecosystem Dynamics and SARA Computing to manage bird tracking data and develop tools for data exploration, analysis and visualization. This wiki is designed to facilitate access to the data and tools within the Virtual Laboratories and to provide a forum for collaboration. <br />
<br />
== UvA-BiTS data == <br />
<br />
:Bird data collected with the UvA-BiTS Bird Tracking System<br />
<br />
Note: We are continually working on improving UvA-BiTS services, including data processing and Virtual Lab functionality. In some cases, this has enabled us to extract more information from old GPS log files which is then incorporated into the database. These changes may result in slight differences when you reprocess historic data. <br />
<br />
::* [[ DB_Model_2014 | e-Ecology data model]]<br />
::* [[ UvAGPS|UvA-BiTS Data Model]] - OLD<br />
::* [[Connecting to the UvA-BiTS database]]<br />
::* [[Useful Queries]]<br />
::* [[Queries for linking bird tracks to wind and flow data]]<br />
<br />
==UvA-BiTS Services==<br />
<br />
* Visualization Tools<br />
:: [[KMZ Generator2]]: <br /><br />
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth. <br />
:: [[Calendar]]: <br /><br />
:::Calendar overview with daily statistics of GPS-tracker<br />
:: [[Flight_Generator]]: <br /><br />
:::This tool produces GPX files from UvA-BiTS tracking data for dynamic [http://www.doarama.com Doarama] visualizations.<br />
:: [[BirdView]]: <br /><br />
:::Access meterological and radar data and overlay with GPS bird tracking data.<br />
:: [[BirdSim]]: <br/ ><br />
:::More dynamic version of BirdView that allows the user to scroll through weather data in time and space.<br />
:: [[KMZ Generator|UvA-BiTS KMZ Generator]]: <br /><br />
:::Produces kmz files of UvA-BiTS bird tracking data to be viewed in Google Earth. <br />
<br />
* [[Dashboard]]: These web pages allow UvA-BiTS users to get an overview of the raw data files of GPS bird tracking devices.<br />
<br />
* [[ Annotation tables ]] for uploading GPS annotation data into your own schema. This service is currently not available.<br />
<br />
== Changes in the Database in 2014 ==<br />
'''Important'''<br />
In the summer of 2014 we have '''re-arranged the administrative''' part of UvA-BiTS (and e-Ecology) in the database. End users may have to adapt their programs the following way:<br />
:* The '''uva_*_limited''' views that used to exist in the gps schema are gone. Most of them should have been replaced with an equivalent '''ee_*_limited counterpart'''. <br />
:* The new ee_*_limited views show a superset of the same information as before, but enhanced with new project information.<br />
:* Some '''column names''' in the ee_*_limited views may differ from the previous views.<br />
:* There are new limited views: <br />
:** gps.ee_'''project'''_limited: shows your projects' information<br />
:** gps.ee_'''shared'''_*_limited: set of data you are able to see related to trackers that a project is sharing with a project you belong to<br />
:* Special attention: gps.uva_device_limited is now replaced by '''gps.ee_tracker_limited'''. For each tracker, you now see which project provides you access to that tracker.<br />
:* Also, '''gps.uva_tracking_limited''' is gone. Instead of that we have gps.ee_tracker_speed_limited, which adds speed columns to the original data. This amounts to a merger of gps.ee_tracking_limited, gps.ee_tracking_speed_3d_limited and gps.ee_tracking_speed_limited.<br />
<br />
==Other e-Ecology Datasets==<br />
<br />
'''ENVIRONMENTAL DATA'''<br />
<br />
::* [[ ECMWF|Weather Forecast Data (ECMWF)]]<br />
::* [[ Elevation|Elevation Data ]]<br />
<br />
'''FLYSAFE'''<br />
<br />
:Bird data created from weather radar data and bird migration models. Financed by the Royal Netherlands Air Force. <br />
<br />
::* [[Connecting to the Flysafe database]]<br />
::* [[GPS|Flysafe GPS Data Model]]<br />
::* [[ RADAR|Radar Data Model]]<br />
::* [[ Wind profile|Wind Profile Data Model]]<br />
::* [[ Fino1|Fino1 Data Model]]<br />
<br />
==Other e-Ecology Services==<br />
[[GPS tracks|FlySafe GPS Bird Track Generator]]: Produce kmz files of ARGOS/GPS bird tracking data from the FlySafe project to be viewed in Google Earth. <br />
<br />
[[ Administration]]: Information about user rights and data processing tasks.<br />
<br />
[[ Bookkeeping]]: Description of tables related to FlySafe data processing and archiving.<br />
<br />
[[ ToDo ]]</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=Flight_Generator&diff=1205Flight Generator2015-01-12T09:50:50Z<p>Stefan verhoeven: moved GPX to Flight Generator</p>
<hr />
<div>== Flight generator ==<br />
<br />
Doarama (http://http://www.doarama.com/) is a GPS track visualization tool. To add a track to Doarama a GPX file is required.<br />
A GPX file of a UvA-BiTS track can be generated using this service.<br />
<br />
The service itself can be found via the Virtual Lab [http://www.uva-bits.nl/virtual-lab/] by selecting "flight generator" or via direct link at [https://services.e-ecology.sara.nl/sw/tool/gpx/ e-ecology website]. This requires special access: You need to have credentials and your IP address should be registered. Please contact grid.support@sara.nl if you have problems getting access.<br />
<br />
=== System Requirements ===<br />
<br />
==== Software Requirements ====<br />
The webservice itself is standard javascript website, and as such is supported by all modern browsers and operating systems.<br />
<br />
==== Hardware requirements ====<br />
The Doarama (http://http://www.doarama.com/) visualization is a visual application, and therefore do require a little bit of graphical power. Fortunately, most modern computers have decent hardware for graphical acceleration, so this should not be a problem. But for example, if the graphical drivers are not installed correctly, or if remote desktop tools are used, performance can be limited. <br />
<br />
Additionally, Doarama require a decent internet connection. They use satellite images as background, and these have to be retrieved from the internet. Furthermore, if large amounts of data are selected in the flight generator, performance issues may arise.<br />
<br />
==== The interface ====<br />
<br />
=== The Submit Interface ===<br />
[[Image:GPXSubmit.png|thumb|right|400px|abc|]]<br />
<br />
==== Select start date and end date ====<br />
<br />
For the begin and end date, there are two options: You can either type a date directly, or you can use the calender button. When you use the calender button, a small window pops up where a desired date can be selected. Here you can either use the arrows, or you can use ctrl up/down and ctrl left/right to change the years and months respectively. If you have selected a year and month, you can click on a day to finalize you date selection. For more detailed time tuning, there are also fields for the hours and minutes. Again, you can either type it, or you can use the drop down to select the desired hour.<br />
<br />
==== Select tracker ====<br />
<br />
To select tracker click on the row. The checkbox should be checked.<br />
<br />
==== Save selection ====<br />
<br />
Users often are interested in the same tracker. It would be tedious to select the same tracker each time. Therefore we added functionality to save and load specific selections.<br />
To save the current selection, press the ''Save selection'' button, choose a name, by default the current time will be used.<br />
To restore a saved selection, press the ''Restored saved selection'' button, double click on the name or press the icon with the arrow.<br />
<br />
As a bonus the last submitted selection is always saved as ''Last used''. This can be used when you want to make the same gpx file again or make small changes without configuring everything again.<br />
<br />
It must be noted that the saved selections are browser specific. So will not show up when using a different computer.<br />
<br />
=== Visualization ===<br />
<br />
To make a Doarama visualization:<br />
<br />
# Download the GPX file from the result page<br />
# Goto http://www.doarama.com and Login<br />
# Click 'Upload' in topleft corner of page<br />
# Select the downloaded GPX file and upload it<br />
# Edit description <br />
# Set 'Activity Type' to 'Fly Bird / Aves'<br />
# Create a visualization<br />
# Press save button</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=GPX&diff=1206GPX2015-01-12T09:50:50Z<p>Stefan verhoeven: moved GPX to Flight Generator</p>
<hr />
<div>#REDIRECT [[Flight Generator]]</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=Flight_Generator&diff=1198Flight Generator2015-01-09T09:13:40Z<p>Stefan verhoeven: Created page with "== Flight generator == Doarama (http://http://www.doarama.com/) is a GPS track visualization tool. To add a track to Doarama a GPX file is required. A GPX file of a UvA-BiTS tra..."</p>
<hr />
<div>== Flight generator ==<br />
<br />
Doarama (http://http://www.doarama.com/) is a GPS track visualization tool. To add a track to Doarama a GPX file is required.<br />
A GPX file of a UvA-BiTS track can be generated using this service.<br />
<br />
The service itself can be found via the Virtual Lab [http://www.uva-bits.nl/virtual-lab/] by selecting "flight generator" or via direct link at [https://services.e-ecology.sara.nl/sw/tool/gpx/ e-ecology website]. This requires special access: You need to have credentials and your IP address should be registered. Please contact grid.support@sara.nl if you have problems getting access.<br />
<br />
=== System Requirements ===<br />
<br />
==== Software Requirements ====<br />
The webservice itself is standard javascript website, and as such is supported by all modern browsers and operating systems.<br />
<br />
==== Hardware requirements ====<br />
The Doarama (http://http://www.doarama.com/) visualization is a visual application, and therefore do require a little bit of graphical power. Fortunately, most modern computers have decent hardware for graphical acceleration, so this should not be a problem. But for example, if the graphical drivers are not installed correctly, or if remote desktop tools are used, performance can be limited. <br />
<br />
Additionally, Doarama require a decent internet connection. They use satellite images as background, and these have to be retrieved from the internet. Furthermore, if large amounts of data are selected in the flight generator, performance issues may arise.<br />
<br />
==== The interface ====<br />
<br />
=== The Submit Interface ===<br />
[[Image:GPXSubmit.png|thumb|right|400px|abc|]]<br />
<br />
==== Select start date and end date ====<br />
<br />
For the begin and end date, there are two options: You can either type a date directly, or you can use the calender button. When you use the calender button, a small window pops up where a desired date can be selected. Here you can either use the arrows, or you can use ctrl up/down and ctrl left/right to change the years and months respectively. If you have selected a year and month, you can click on a day to finalize you date selection. For more detailed time tuning, there are also fields for the hours and minutes. Again, you can either type it, or you can use the drop down to select the desired hour.<br />
<br />
==== Select tracker ====<br />
<br />
To select tracker click on the row. The checkbox should be checked.<br />
<br />
==== Save selection ====<br />
<br />
Users often are interested in the same tracker. It would be tedious to select the same tracker each time. Therefore we added functionality to save and load specific selections.<br />
To save the current selection, press the ''Save selection'' button, choose a name, by default the current time will be used.<br />
To restore a saved selection, press the ''Restored saved selection'' button, double click on the name or press the icon with the arrow.<br />
<br />
As a bonus the last submitted selection is always saved as ''Last used''. This can be used when you want to make the same gpx file again or make small changes without configuring everything again.<br />
<br />
It must be noted that the saved selections are browser specific. So will not show up when using a different computer.<br />
<br />
=== Visualization ===<br />
<br />
To make a Doarama visualization:<br />
<br />
# Download the GPX file from the result page<br />
# Goto http://www.doarama.com and Login<br />
# Click 'Upload' in topleft corner of page<br />
# Select the downloaded GPX file and upload it<br />
# Edit description <br />
# Set 'Activity Type' to 'Fly Bird / Aves'<br />
# Create a visualization<br />
# Press save button</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=File:GPXSubmit.png&diff=1197File:GPXSubmit.png2015-01-09T09:06:29Z<p>Stefan verhoeven: Doarama generator/flight generator/ gpx file submit form</p>
<hr />
<div>Doarama generator/flight generator/ gpx file submit form</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=Tracker_Calendar&diff=1196Tracker Calendar2015-01-09T08:56:42Z<p>Stefan verhoeven: /* Finding outliers */</p>
<hr />
<div>== The UvA-BiTS Tracker calendar==<br />
<br />
The UvA-BiTS GPS tracking data can aggregated for each day. This tracker calendar shows daily statistics or metrics of tracker as a calendar heatmap.<br />
It can be used to find days when something interesting happened or to find repeating patterns.<br />
<br />
The service itself can be found via the Virtual Lab [http://www.uva-bits.nl/virtual-lab/] by selecting "tracker calendar" or via direct link at [https://services.e-ecology.sara.nl/sw/tool/calendar/ e-ecology website]. This requires special access: You need to have credentials and your IP address should be registered. Please contact grid.support@sara.nl if you have problems getting access.<br />
<br />
=== Metrics ===<br />
<br />
The following metrics of a tracker are calculated for each day:<br />
<br />
* Nr. of GPS measurements<br />
* Nr. of accelerometer measurements<br />
* 2D distance travelled (km): Distance of line which consists of GPS measurements from midnight till midnight, altitude is ignored.<br />
* Maximum altitude (m): Highest absolute altitude<br />
* Average altitude (m): Average absolute altitude<br />
* Minimum altitude (m): Lowest absolute altitude<br />
* Maximum temperature (°C)<br />
* Average temperature (°C) <br />
* Minimum temperature (°C) <br />
* Minimum voltage battery (V): Lowest battery voltage<br />
* Maximum interval between GPS measurements (hh:mm:ss)<br />
* Minimum interval between GPS measurements (hh:mm:ss) <br />
<br />
=== System Requirements ===<br />
<br />
==== Software Requirements ====<br />
The webservice itself is standard javascript website, and as such is supported by all modern browsers and operating systems.<br />
<br />
=== The Submit Interface ===<br />
[[Image:CalendarSubmit.png|thumb|right|400px|abc|]]<br />
<br />
==== Select start date and end date ====<br />
<br />
For the begin and end date, there are two options: You can either type a date directly, or you can use the calender button. When you use the calender button, a small window pops up where a desired date can be selected. Here you can either use the arrows, or you can use ctrl up/down and ctrl left/right to change the years and months respectively. If you have selected a year and month, you can click on a day to finalize you date selection. For more detailed time tuning, there are also fields for the hours and minutes. Again, you can either type it, or you can use the drop down to select the desired hour.<br />
<br />
==== Select tracker ====<br />
<br />
To select tracker click on the row. The checkbox should be checked.<br />
<br />
==== Save selection ====<br />
<br />
Users often are interested in the same tracker. It would be tedious to select the same tracker each time. Therefore we added functionality to save and load specific selections.<br />
To save the current selection, press the ''Save selection'' button, choose a name, by default the current time will be used.<br />
To restore a saved selection, press the ''Restored saved selection'' button, double click on the name or press the icon with the arrow.<br />
<br />
As a bonus the last submitted selection is always saved as ''Last used''. This can be used when you want to make the same calendar again or make small changes without configuring everything again.<br />
<br />
It must be noted that the saved selections are browser specific. So will not show up when using a different computer.<br />
<br />
=== The Result Interface ===<br />
[[Image:CalendarResult.png|thumb|right|400px|abc|]]<br />
<br />
In the top left of the page you can see which tracker, date range and metirc has been selected.<br />
In the top right of the page you can see the lowest and highest value for the currently selected tracker.<br />
The calendar is itself:<br />
<br />
* White background: no data for any metric on that day or day is outside selected date range<br />
* Gray background: no data for the selected metric on that day, but another metric has data<br />
* Moving mouse over a day will show the date and value of the metric<br />
<br />
On the bottom of the page you can download a csv file with the data shown in the calendar.<br />
<br />
==== Change metric ====<br />
<br />
The combobox at the top left corner of the page can be used to change to a different metric.<br />
<br />
==== Finding outliers ====<br />
<br />
The sliders in the top right corner of the page can be used to find days which are outliers.<br />
<br />
Dragging the left slider will change the minimum to a higher value, all days below this value will be colored 'blue'.<br />
<br />
Dragging the right slider will change the maximum to a lower value, all days above this value will be colored 'red'.<br />
<br />
Dragging the sliders will rescale the colors making it easier to see small changes.<br />
<br />
==== Change colors ====<br />
<br />
The default color range is lowest is light green and highest is dark green.<br />
The colors can be changed by <br />
<br />
# Press the 'Colors...' button in the top right corner of the page.<br />
# For the color you want to change press the '&or;' button<br />
# Pick a color<br />
# Press the 'Select Color' button</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=Tracker_Calendar&diff=1195Tracker Calendar2015-01-09T08:53:51Z<p>Stefan verhoeven: Create tracker calendar help page</p>
<hr />
<div>== The UvA-BiTS Tracker calendar==<br />
<br />
The UvA-BiTS GPS tracking data can aggregated for each day. This tracker calendar shows daily statistics or metrics of tracker as a calendar heatmap.<br />
It can be used to find days when something interesting happened or to find repeating patterns.<br />
<br />
The service itself can be found via the Virtual Lab [http://www.uva-bits.nl/virtual-lab/] by selecting "tracker calendar" or via direct link at [https://services.e-ecology.sara.nl/sw/tool/calendar/ e-ecology website]. This requires special access: You need to have credentials and your IP address should be registered. Please contact grid.support@sara.nl if you have problems getting access.<br />
<br />
=== Metrics ===<br />
<br />
The following metrics of a tracker are calculated for each day:<br />
<br />
* Nr. of GPS measurements<br />
* Nr. of accelerometer measurements<br />
* 2D distance travelled (km): Distance of line which consists of GPS measurements from midnight till midnight, altitude is ignored.<br />
* Maximum altitude (m): Highest absolute altitude<br />
* Average altitude (m): Average absolute altitude<br />
* Minimum altitude (m): Lowest absolute altitude<br />
* Maximum temperature (°C)<br />
* Average temperature (°C) <br />
* Minimum temperature (°C) <br />
* Minimum voltage battery (V): Lowest battery voltage<br />
* Maximum interval between GPS measurements (hh:mm:ss)<br />
* Minimum interval between GPS measurements (hh:mm:ss) <br />
<br />
=== System Requirements ===<br />
<br />
==== Software Requirements ====<br />
The webservice itself is standard javascript website, and as such is supported by all modern browsers and operating systems.<br />
<br />
=== The Submit Interface ===<br />
[[Image:CalendarSubmit.png|thumb|right|400px|abc|]]<br />
<br />
==== Select start date and end date ====<br />
<br />
For the begin and end date, there are two options: You can either type a date directly, or you can use the calender button. When you use the calender button, a small window pops up where a desired date can be selected. Here you can either use the arrows, or you can use ctrl up/down and ctrl left/right to change the years and months respectively. If you have selected a year and month, you can click on a day to finalize you date selection. For more detailed time tuning, there are also fields for the hours and minutes. Again, you can either type it, or you can use the drop down to select the desired hour.<br />
<br />
==== Select tracker ====<br />
<br />
To select tracker click on the row. The checkbox should be checked.<br />
<br />
==== Save selection ====<br />
<br />
Users often are interested in the same tracker. It would be tedious to select the same tracker each time. Therefore we added functionality to save and load specific selections.<br />
To save the current selection, press the ''Save selection'' button, choose a name, by default the current time will be used.<br />
To restore a saved selection, press the ''Restored saved selection'' button, double click on the name or press the icon with the arrow.<br />
<br />
As a bonus the last submitted selection is always saved as ''Last used''. This can be used when you want to make the same calendar again or make small changes without configuring everything again.<br />
<br />
It must be noted that the saved selections are browser specific. So will not show up when using a different computer.<br />
<br />
=== The Result Interface ===<br />
[[Image:CalendarResult.png|thumb|right|400px|abc|]]<br />
<br />
In the top left of the page you can see which tracker, date range and metirc has been selected.<br />
In the top right of the page you can see the lowest and highest value for the currently selected tracker.<br />
The calendar is itself:<br />
<br />
* White background: no data for any metric on that day or day is outside selected date range<br />
* Gray background: no data for the selected metric on that day, but another metric has data<br />
* Moving mouse over a day will show the date and value of the metric<br />
<br />
On the bottom of the page you can download a csv file with the data shown in the calendar.<br />
<br />
==== Change metric ====<br />
<br />
The combobox at the top left corner of the page can be used to change to a different metric.<br />
<br />
==== Finding outliers ====<br />
<br />
The sliders in the top right corner of the page can be used to find days which are outliers.<br />
Dragging the left slider will change the minimum to a higher value, all days below this value will be colored 'blue'.<br />
Dragging the right slider will change the maximum to a lower value, all days above this value will be colored 'red'.<br />
<br />
==== Change colors ====<br />
<br />
The default color range is lowest is light green and highest is dark green.<br />
The colors can be changed by <br />
<br />
# Press the 'Colors...' button in the top right corner of the page.<br />
# For the color you want to change press the '&or;' button<br />
# Pick a color<br />
# Press the 'Select Color' button</div>Stefan verhoevenhttps://wiki.e-ecology.nl/index.php?title=File:CalendarResult.png&diff=1194File:CalendarResult.png2015-01-09T08:33:56Z<p>Stefan verhoeven: Tracker calendar result page screenshot</p>
<hr />
<div>Tracker calendar result page screenshot</div>Stefan verhoeven