2021/05/04 - Web services and Rest-APIThese changes made this post obsolete. A new version entitled Données de marée au Canada avec Python - nouvelle interface REST (mai 2021), soon to be translated, is available. It contains a Python script that uses the new REST interface which can upload the desired data in JSON format in response to a simple HTTP request.
As of 12:00 pm EDT on Monday, May 3rd, 2021, the old Canadian Hydrographic Service WDS services (observations and predictions) have been decommissioned. Please refer to the web services page for information on using the new REST-API for direct access to official water level data and other available services.
This page will remain because it contains difficult to find information about the use of SOAP and WSDL technologies in Python. It is unfortunate that its working example no longer works.
While looking out across the river at noon, I was wondering if the tide was coming in or going out. There is no obvious way to determine if the tide is ebbing or flowing except perhaps comparing the current water level with that seen on a photograph taken an hour before. Were the house on the eastern coast of New Brunswick (about 70 km away), I could probably tell by the direction of flow of the water because the tidal range can be as much as 17 metres at some points on the Bay of Fundy. However, "our" river, the Shediac River, flows into Shediac Bay on the Northumberland Strait and the tidal range in that bay is little more than a meter. Here is the forecast tides for today.
Times and Heights for High and Low Tides
Shediac Bay, 2019-05-18 |
|
---|---|
Time | Height (m) |
03:44 | 1.10 |
09:35 | 1.20 |
16:34 | 0.50 |
As can be seen the water level was falling by 10 cm per hour around noon (linear approximation, which is wrong). It is very doubtful that I would be able to perceive a drop of 8 mm in the level of the river over a five-minute interval at a distance of 30 metres. Actually, the drop would be smaller yet because the tidal waters must travel up the river 4 or 5 km to get to the house so that the tidal range is probably considerably less in front of the house.
I could try to follow a leaf or other floating object in the river. However, light winds can influence the flow of surface water and often overcome the direction of the tidal water. Really, the only foolproof methods that I can imagine to answer the question are:
- lookup the tide tables for Shediac Bay,
- measure the height of the water in the river at regular intervals saving the results.
Both these solutions interest me, but I will talk about the first option here.

When I was a young man, I learned about tides and the correct method to interpolate data from the tide tables. Back then, that was done by fitting a sine curve to the forecast hourly data. If I recall correctly, navigators used a sine table to interpolate the data while engineers might have used a slide rule. That shows how old I am, because nowadays an app which fits some type of spline to the data is probably what everone uses.
While annual tide tables are still printed and can be had for a price (see the image on the right), I am too cheap to spend $6.50 CDN for Canadian Tide and Current Tables, Volume 2: Gulf of St. Lawrence. Indeed, does anyone still buy a paper edition when the data is available for free from the Canadian Hydrographic Service of the Department of Fisheries and Oceans, Canada? There is a toll-free telephone service: Water Levels at Your Fingertips! and a web page for each station in Canada; here are the Shediac Bay predictions for 2019. A full year of predictions is perhaps more than needed at a point in time. There is another page at Fisheries and Oceans that shows predicted times and height of high and low tides and the predicted hourly water heights for 7 days: Shediac Bay * (#1805), 7 days Tidal Predictions. The data can be displayed in a graph, a table or as text. It would not be too difficult to "scrape" the data from the text version. In addition, it is possible to get observed water levels in Shediac Bay for the 24 hours at a time in 15-minute intervals from the same page.
It was not difficult to work out the format of the URL request.
Param | Value | |
---|---|---|
LANG | eng | fra | English or French |
TYPE | 0 | 1 | Predictions or observations (#) |
YYYY | 4 digit year | |
MM | 2 digit month | |
DD | 2 digit day | |
ID | station id number | See Index of Sites |
TZ | UTC | NDT | NST | ADT | AST | EDT | EST | CDT | CST | MDT | MST | PDT | PST | |
MODE | 0 | 1 | 2 | Graph, table or text |
(#) Observations are available only for those stations such as Shediac Bay that are marked with an asterisk (*) in the Index of Sites. The local time in the six Canadian time zones is defined in relation to the Coordinated Universal Time (UTC).
Daylight time | Standard time | ||||
---|---|---|---|---|---|
Zone | Local time | Zone | Local time | ||
Newfoundland | NDT | UTC -2:30 | NST | UTC -3:30 | |
Atlantic | ADT | UTC -3:00 | AST | UTC -4:00 | |
Eastern | EDT | UTC -4:00 | EST | UTC -5:00 | |
Central | CDT | UTC -5:00 | CST | UTC -6:00 | |
Mountain | MDT | UTC -6:00 | MST | UTC -7:00 | |
Pacific | PDT | UTC -7:00 | PST | UTC -8:00 |
Strangely, Fisheries and Oceans defines
Atlantic Standard Time as
AST (Z+4) Atlantic Standard Time
, where Z =
Zulu time is the military name for GMT (Greenwich Mean Time, the UTC
predecessor). Shediac is west of Greenwich and local time is Z-4. No big
deal, the local times shown in the tables, graphs, etc. on the site are
correct.
The URL to obtain a graph of water levels for 7 days starting on May 18, 2019 in Shediac Bay in French using Atlantic Daylight Time is:
Here is the relevant part of the output.
The idea of doing data scraping is not very appealing. I was hoping to find a well-defined REST API to get the desired data in the same fashion as weather information can be obtained (see The Yahoo! Weather API with Free Pascal and More Weather API). Unlike for weather information, I could not find a site that offers (near) "universal" tide data free of charge for tinkerers like myself. Fortunately, and not surprisingly given the above, the Canadian Hydrographic Service (CHS) does give free access to its water levels web services as long the Licence agreement is respected. Many sites with tide information for Shediac Bay do not mention CHS. Either they are obtaining their information from another source, which I have not been able to identify, or they are flaunting the licence which clearly states:
4. The User accepts the following conditions governing the use of the Data and the Service:
- The copyright in the Data is the property of Her Majesty the Queen in right of Canada, as represented by the Minister of Fisheries and Oceans, on behalf of the Canadian Hydrographic Service. The copyright may NOT be sold, licensed, leased, assigned or given to a third party.
- The User has the right to use the Service, reproduce and distribute the Data, and to manufacture or cause to be manufactured and sell or license or cause to be sold or licensed files, products or publications which derive from or incorporate or by any other means use the Data and the Service, in whole or in part (“Derivative Product”), and to sub-licence the right to use and reproduce the Data and the Service, PROVIDED:
- the User acknowledges CHS as the copyright holder of the Data on all reproductions of the Data;
- where any of the Data is contained within a Derivative Product, the User includes in a prominent location on such Derivative Product the following notice: “This product has been produced by or for [insert User's corporate name] and includes data and services provided by the Canadian Hydrographic Service of the Department of Fisheries and Oceans. The incorporation of data sourced from the Canadian Hydrographic Service of the Department of Fisheries and Oceans within this product does NOT constitute an endorsement by the Canadian Hydrographic Service or the Department of Fisheries and Oceans of this product.”
- ...
Since I have already reproduced CHS tide data, I hope that including the above respects the agreement.
At first, it looked like it would be difficult to access the data from the CHS website because it used technologies that seemed to me old and arcane.
[the] web services use SOAP and XML as their communication protocol and the English language for method calling and data exchange. Each service has an XML description readable in WSDL (i.e. https://ws-shc.qc.dfo-mpo.gc.ca/predictions?wsdl). This description is useful for automatically generating code (with tools such as WSDL2Java from Apache Axis) to communicate with the services.
No REST API? No JSON formatted output? WSDL? I am out of my depth. However there is a reference to a cookbook on sourceforge in the Technical specifications for accessing Water Level Web Services. The cookbook contains a 5 line example in Python.
That was enough to get started. The SOAPpy
module is not
installed in either version 2.7 of Python or in version 3.6 bundled with
Ubuntu 18.04. Before installing it, it seemed wise
to look into other modules and I opted to install Zeep
,
a SOAP client only, because it is actively maintained and available for both
versions of Python. The last updates to SOAPpy
go back five
years and the module is more ambitious since it also implements a SOAP server.
Here is the CHS example using Zeep
and its output. (I now
use Suds
as explained at the end of the post.)
That bounding region occupies a big chunk of the earth's surface. I
am not sure what it represents, certainly not Canada, but it does not
matter; the important point is that data was obtained from the CHS
site. The important function for my purpose is called search
.
It is documented with a couple of examples on pages 7 and 8 of Technical specifications for accessing Water Level Web
Services. With that information I was able to cobble a script that
displays the times and heights in metres of the high and low tide in
Shediac Bay for a given day.
Downloadable version of the script: tide.py.
Once again, I make no claim about the quality of my Python scripts. But it does work:
Of course this is only a proof of concept script. The hard part remains to be done:
- How to incorporate this into my home automation system. There is no ready-made high/low tide virtual sensor in Domoticz as far as I can tell. Perhaps a text sensor showing the time and height of the previous high or low tide and the time and height of the next high or low tide event would be sufficient to answer my initial question: is the tide flowing or ebbing. Perhaps a simple water level measure with an indication if it is lower or higher than the previous measure would suffice.
- How to measure and log the true water level of the river in front of the house because high and low tides occur later than in the bay. By the same token, water levels should not fluctuate as much upstream in the river as in the bay. With that information, I could perhaps derive adjustment factors to calculate the local water levels from the predicted levels in Shediac Bay.
- How to do this in Free Pascal. This is not a totally idle make work project. I used Pascal to obtain information from Yahoo! Weather, Dark Sky, MET Normay, etc and it would be fitting to combine weather and tide data.
The good news with respect to the first point is that
tide.py
works on a Raspberry Pi running Raspbian. The bad news is that installing Zeep
was a mess. I tried a lot of things to install the module and I will have to
start from a fresh copy of the OS to try to figure out what really needs to
be done to install the module. This is where things sometimes break down,
when for unknown reasons, it seems impossible to install the package
correctly a second time. Time will tell.
It was relatively easy to install Zeep
on Raspbian to use tide.py
. I just had to follow the precise instructions written by Michael van Telligent, the main
author of the module, after installing the libxslt1.1
library
which required by lxml
.
Modules pip
and its counterpart for Python 3,
pip3
, are not contained in the Stretch Lite
ofRaspbian. They may be included in the full version of
Stretch
. In contrast, pip
is automatically
available in each virtual Python environment.
No matter what I tried, I could not install lxml
in Armbian 5.83 on an Orange Pi Zero. The last option was to
attempt to compile the module from its source, but instead of doing that I
found an easier solution that consists of using the module Suds
rather than Zeep
. The original module is old (circa end of 2007
with a last update on April 31, 2009), but there are many forks among which
suds-community
seems the most active. It was last updated was on
February 7th. Its installation could not be simpler.
The script tide.py
will function exactly as before with a
when the line
is changed to