My adventures in collecting and using home weather data

[jollystomper]

I originally posted this in the Blow That Dough! 2020 thread. A few people were interested in hearing more information, so here goes the nerdiness... you have been warned.

One of my projects of interest is tracking temperature/humidity inside and outside of our home. My requirements were to monitor at least 6 different areas and feed the data into my home computing center to apply my desired analytics, integrate with other data, or whatever else came to mind.

I wanted to start simple – it is easy to start simple and choose to make it complex, than the other way around. I also wanted to start relatively cheap.

I debated the "build your own" option using Raspberry Pis, sensors, and connecting components, or a more "out of the box" solution. But in true "Blow That Dough" mindset, I realized, "why not both?" My starting point ls likely not the “optimal” or “best” option, it is just the one I chose as a quick, simple way to start leaning and implementing this stuff.

The simple "out of the box" solution I uses the Ambient Weather WS-3000 monitor with 8 sensors (description at https://ambientweather.com/amws3000x8.html). The sensors run on batteries, which gives more flexibility for where to place them. The WS-3000 can record the collected data on a SD card. So my initial thought was to add an SD card, and either (a) occasionally take it out to copy the data, or (b) use the Ambient software for this monitor to sign on and copy/paste the data as needed.

One challenge is that it keeps separate data files for each sensor. Unfortunately, the device does not give visibility to the SD card as a drive when connected via USB.

However, knowing that I am not the sharpest nor earliest tool in the shed , I searched to find if others has done anything to improve automating the data collection. Sure enough, on GitHub there are a couple of packages one can try. One is node.js based, one is python, I used the node.js package (https://github.com/EpicVoyage/ambientweather-ws3000) as it seem easier to get it up and running.

The package requires Linux to run on. Looking at the source code it finds the ws3000 based on messages emitted in the Linux dmesg command when the ws3000 is connected to the USB port. It then sends the appropriate commands (not documented but from network tracing they figured out the command format) to retrieve temperature and humidity data.

Having worked with node.js at MegaCorp (though not having touched it since retiring I did need a little refresher, thank you internet ), I was able to quickly set up a process where, every 30 minutes, Linux runs a node.js JavaScript program to retrieve the current data for all of the sensors, do any calculations I need (e.g. the sensor temperature data is in Celsius so I calculate the Fahrenheit equivalent) and write it to a csv file. Another node program reads the csv file and extracts data for use by a JavaScript program that uses the graph.js Javascript package to generate a quick-and-dirty line chart. Another node process runs as a web server that I can use to display the line chart (and any other charts I might choose to create).

I am starting to analyze the data, and already see some interesting correlations for starters that many lead to me "tweaking" some aspects of home thermostat/dehumidifier/attic fan just to see if things change.

I have decided I want to do more customized queries (e.g, retrieve data for particular sensors within a given time range), so, in addition to saving the data as a .csv file, I also store it in a sqlite database (yeah, I could just use sqlite and the .csv file, but for now, storage is cheap).
Also, this gives me incentive to educate myself on writing an Android app, so that I can get the data on my phone.

This has will give me time to play with the "build your own" option as a self-education project, and feel not feeling "rushed" to get something working. I’ll likely look at some hardware architecture around a Raspberry Pi, wife controllers, and sensors, but I have to do more (and leisurely research) on this. It will be interesting to see how the costs compare..., but no worries, Blow that Dough!

Anyway, as I said, not the best or most efficient way (a picture would be a lot clearer than my ramblings). But for me, sometimes the journey by itself can be a learning experience, so I do not mind hacking around and making the journey longer (and hey, I’m FIRE, so no need to rush it!)

 

[JoeWras]

Jolly stomper. I'm replying to mark this thread. I have huge interest in this, starting with my own Turbo-C version I wrote for a Peer Bros station back in 96.

I don't have time to expand on it right now. Hopefully tomorrow. I'd like to chat more. Later...

 

[Yoheadden]

I have thought for years about getting a weather station to track the weather conditions at my house. The “local” weather never quite seems to have in zeroed in. Plus, we have a pretty open view and I am curious about the wind gusts that we get hit with. An out of the box system would probably meet my needs, but it is interesting to read how next lvl you have taken it.

 

[jollystomper]

I have thought for years about getting a weather station to track the weather conditions at my house. The “local” weather never quite seems to have in zeroed in. Plus, we have a pretty open view and I am curious about the wind gusts that we get hit with. An out of the box system would probably meet my needs, but it is interesting to read how next lvl you have taken it.

Now you have given me more data to try to collect .


With the openweather api (https://openweathermap.org, free api key for the low volumes of requests I make), I can pull their metrics on the current weather conditions for my latitude and longitude, including wind info (steady, direction, gust).


I can start compare what it reports for temperature and humidity to what i am getting. The sensors I have do not measure wind, so perhaps - since I will only need one - that is where I can focus my "build your own" efforts on - getting wind data from my house into the home computing center.

 

[Bamaman]

I'm just a simple man.

If the humidity gets too high or if I get hot, I turn on the a/c. If I get cold, I turn on the heat.

 

[kgtest]

I'm just a simple man.

If the humidity gets too high or if I get hot, I turn on the a/c. If I get cold, I turn on the heat.

SAME! I actually run with the idea, always OFF unless necessary. Low carbon footprint. I could see how I get caught up with the coding to get all that data etc... would be cool to work on for a bit.

node.js... you can make a career outta that! It's funny a lot of people want me to have that experience in my field, but I don't actually write code for UI very often... especially code that draws/displays the UI...my role is more test and check that it meets customer specs.

 

[JoeWras]

I’ll likely look at some hardware architecture around a Raspberry Pi, wife controllers, and sensors, but I have to do more (and leisurely research) on this.

 

[JoeWras]

Hi jollystomper.

Sounds like you have a great project out there. Don't listen to those two "simple men" posting above. Some of us love the complexity of watching everything about the weather/temperature either inside, outside or both.

I'm currently on hiatus with my project. Life got in the way during the early 2000s. I had been on quite a run. But then my parents got sick, work became way too challenging (plenty to learn there, didn't need an outside project) and in my spare time I needed to just do physical things such as exercise or gardening.

Now that I'm retired, I'd like to take it up again, once I get done with my mega-drainage and garden project (probably until next summer). I spent the Covid lockdown time doing photo archival, otherwise that would have been a good time to goof around with it.

My setup is a Peet Bros Ultimeter 2000 (https://www.peetbros.com). I've had it since 1993! Amazing reliability. The problem is that Peet Bros is basically drifting out of business. I think the guy who owns it retired to FL and basically just runs the business with his old inventory as a hobby. I'm not sure there is a future. At least he is very responsive to the small base still out there. He even gave me forgiveness when I broke my station by shorting out some pins on the dataport, way back in 1995, and upgrading to the latest at the time. It has been running solidly for 25 years since!

The cool thing is that Peet Bros publishes their data flow. It comes out as a serial port (there's a hint as to how old it is). So there was no guessing or reverse engineering.

I first wrote a "C" program on a dedicated DOS computer. I had massive data I gathered in table form.

Later, I converted to C++, and ran in the background on Windows. In the early 2000s, I ported it as a Linux application running on a very simple little computer with a USB to Serial adapter. At this time, I also wrote a daemon that ftp'd live data to my web site. I could watch the data remotely.

FTP is insecure, and it eventually failed as the web hosting service didn't allow it. Since then, it is off-line. I need to dust off the whole thing and nearly start from scratch. I'd love to push the data to an on-line database and have a more sophisticated page on my web site. Like you, I'm also interested in creating an Android app.

Now that I'm 2+ years out of retirement, I'm feeling the itch to program again. I had no stomach for it for quite a while. I'd like to exercise my python, and maybe even delve into node.js. I also have a raspberry pi which this would be perfect for.

The question is do I buy new, possibly the Ambient, or continue with Peet? I'd love to contribute to github. There's a bit on Github for Peet, but the community has pretty much dried up. I might keep it generic in general and start with Peet to generate my data, with possible additions of different stations on the HW back end generating to the same data set format.

I guess you could say you are not alone. WX data geeks unite!

 

[jollystomper]

Hi jollystomper.

Sounds like you have a great project out there. Don't listen to those two "simple men" posting above. Some of us love the complexity of watching everything about the weather/temperature either inside, outside or both.

I'm currently on hiatus with my project. Life got in the way during the early 2000s. I had been on quite a run. But then my parents got sick, work became way too challenging (plenty to learn there, didn't need an outside project) and in my spare time I needed to just do physical things such as exercise or gardening.

Now that I'm retired, I'd like to take it up again, once I get done with my mega-drainage and garden project (probably until next summer). I spent the Covid lockdown time doing photo archival, otherwise that would have been a good time to goof around with it.

My setup is a Peet Bros Ultimeter 2000 (https://www.peetbros.com). I've had it since 1993! Amazing reliability. The problem is that Peet Bros is basically drifting out of business. I think the guy who owns it retired to FL and basically just runs the business with his old inventory as a hobby. I'm not sure there is a future. At least he is very responsive to the small base still out there. He even gave me forgiveness when I broke my station by shorting out some pins on the dataport, way back in 1995, and upgrading to the latest at the time. It has been running solidly for 25 years since!

The cool thing is that Peet Bros publishes their data flow. It comes out as a serial port (there's a hint as to how old it is). So there was no guessing or reverse engineering.

I first wrote a "C" program on a dedicated DOS computer. I had massive data I gathered in table form.

Later, I converted to C++, and ran in the background on Windows. In the early 2000s, I ported it as a Linux application running on a very simple little computer with a USB to Serial adapter. At this time, I also wrote a daemon that ftp'd live data to my web site. I could watch the data remotely.

FTP is insecure, and it eventually failed as the web hosting service didn't allow it. Since then, it is off-line. I need to dust off the whole thing and nearly start from scratch. I'd love to push the data to an on-line database and have a more sophisticated page on my web site. Like you, I'm also interested in creating an Android app.

Now that I'm 2+ years out of retirement, I'm feeling the itch to program again. I had no stomach for it for quite a while. I'd like to exercise my python, and maybe even delve into node.js. I also have a raspberry pi which this would be perfect for.

The question is do I buy new, possibly the Ambient, or continue with Peet? I'd love to contribute to github. There's a bit on Github for Peet, but the community has pretty much dried up. I might keep it generic in general and start with Peet to generate my data, with possible additions of different stations on the HW back end generating to the same data set format.

I guess you could say you are not alone. WX data geeks unite!

Thanks! you have been at this a lot longer than me, great!

My "frugal, DIY" side says to keep going with the Peet. You have it working, so it might be quicker. But, my "Blow My Dough" side says why look at something more, like the Ambient or something else. Either, you will have fun.

Here is a picture of my simple web page I am using now:

 

[JoeWras]

What's the latest?

Waking up an old thread. Why start new when we can recycle?

First to jollystomper: how is the Ambient still doing?

To everyone: any weather station suggestions out there?

My 29 year old Peet Brothers (29 years!!!!!) is still going strong. The humidity sensor is shot, but it still gives accurate outdoor rain and temperature data. Incredible. The guy who made these (or some successor) still sells them out of their basement, but it is old tech. No good web integration.

I had a Netatmo that just decided to die after few years. So it is time to look for something modern.

"Tempest" is all the rage but people complain about the rain sensor. That's really want I want. Rain and temp. The wind is just a nice to have, and frankly I don't have the correct exposure anyway.

Leaning to Ambient. Change my mind.

 

[EQMAN]

I have used the Weather Station from Davis Insturments (Vantage Pro2). A bit expensive, but very reliable. I mounted it on a telescoping flagpole with the wind instruments about 25 feet and the base station about 6 feet. The data flows to Wunderground and Davis WeatherLInk sites as well as a console I have in the house. Weatherlink lets me store historial data as long as I keep my annual subscription and it is updated ever 2.5 seconds vs. Wunderground that is every 15 minutes and only shows current conditions. I had the prior model but got the new model after 20 years when I moved and did an new installation. Having it on Wunderground allows my neighbors to also see what the current conditions are in our neighborhood and some of them really enjoy that. I can also check either site anywhere I have an internet connection.

 

[JoeWras]

Davis used to be the standard before everything got smartphone integrated. I always pined for a Davis. I'll put them on my look at list.

 

[zinger1457]

I also have a Davis weather station (Vantage Vue), I've had it up since 2010 and it's worked flawlessly. Living in AZ it's nice to be able to get YTD rain totals at the house, currently at 19.22", very good year for us. Suppose to rain next week so might make 20", the most since I've been monitoring.

 

[ocean view]

I installed an Ambient Weather WS-2902C last February and I couldn’t be more pleased. Cost-wise, it was much cheaper than the Davis setups, plus no ongoing subscription costs. I’m just am armchair weather geek, so I don’t need the professional grade equipment.

Data-wise, I have all of the information I could want on my countertop display or on my phone/computer. It seems like every week I learn of a new data point that it collects that I wasn’t previously aware of.

 

[Ronstar]

I need to get something like this. I have a weather api that I now use with a raspberry pi/ python coding to display published weather data on a 19” monitor in my shop. I heard that the api is expiring in 2023 and that the current provider will no longer be providing the service.

I’d like to collect my own weather data and display it preferably on my shop monitor in addition to either a small display or on my computer in my office. And d like to display forecast data with it.

So I’m looking for a weather station to be able to display in 2 locations along with forecasts.

Do any of you weather station users do something like this?

 

[JoeWras]

I need to get something like this. I have a weather api that I now use with a raspberry pi/ python coding to display published weather data on a 19” monitor in my shop. I heard that the api is expiring in 2023 and that the current provider will no longer be providing the service.

I’d like to collect my own weather data and display it preferably on my shop monitor in addition to either a small display or on my computer in my office. And d like to display forecast data with it.

So I’m looking for a weather station to be able to display in 2 locations along with forecasts.

Do any of you weather station users do something like this?

It looks like Ambient can do this, either by buying another console, or DYI programs. Lots of support out there for DYI with this device.

 

[Ronstar]

It looks like Ambient can do this, either by buying another console, or DYI programs. Lots of support out there for DYI with this device.

Thanks for the info - their website and some YouTube videos sure make it look like it would do what I want it to

 

[ocean view]

I need to get something like this. I have a weather api that I now use with a raspberry pi/ python coding to display published weather data on a 19” monitor in my shop. I heard that the api is expiring in 2023 and that the current provider will no longer be providing the service.

I’d like to collect my own weather data and display it preferably on my shop monitor in addition to either a small display or on my computer in my office. And d like to display forecast data with it.

So I’m looking for a weather station to be able to display in 2 locations along with forecasts.

Do any of you weather station users do something like this?

https://ambientweather.net/

The Ambient weather station data is uploaded to this site, so if you have an internet connection and a browser in your shop, you can view all the data you want. You can determine if it is public or not.

 

[jollystomper]

First to jollystomper: how is the Ambient still doing?

Ambient is still running fine. It has been happily connecting data now for 2+ years. I added an additional sensor without a hitch, right now I am monitoring 7 locations in and outside the house.

 

[Sunset]

https://ambientweather.net/

The Ambient weather station data is uploaded to this site, so if you have an internet connection and a browser in your shop, you can view all the data you want. You can determine if it is public or not.

Thanks, I looked and found a station a few miles from my house, which is close enough to be accurate enough for me.
Could be more accurate than the tv stations for rain , temp , etc..

 

[braumeister]

https://ambientweather.net/

The Ambient weather station data is uploaded to this site, so if you have an internet connection and a browser in your shop, you can view all the data you want. You can determine if it is public or not.

Appears to be very similar to the Weather Underground network.

 

[JoeWras]

Appears to be very similar to the Weather Underground network.

And you can also put your Ambient on the weather underground.

I've decided to go with Ambient. It has good nerd stuff.

Many thanks to jollystomper for starting this thread and also for everyone's input.

 

[zinger1457]

Wasn't familiar with Ambient Weather but from looking over their network coverage I see I have 3 neighbors with a station connected. Makes me wonder why I would need one when I have access to all of their live weather information.

 

[ocean view]

Wasn't familiar with Ambient Weather but from looking over their network coverage I see I have 3 neighbors with a station connected. Makes me wonder why I would need one when I have access to all of their live weather information.

I had the same thoughts, but after monitoring the nearby weather stations for a few months I got the itch for my own toy.

By the way, the Davis folks also have a public network, so if you want to compare data, need something more precise or just want to nerd out, check out their app/website.

 

[street]

I have a friend that has Davis vantage vue wireless. He has no complaints and better wireless range.