Philip P. Ide

Author, programmer, science enthusiast, half-wit.
Life is sweet. Have you tasted it lately?

User Tools

Site Tools


Table of Contents

Raw Weather Data

I have (at last) managed to get hold of all the Mars Weather data that's been missing from my dataset. In fact, I've got hold of so much (over 110Gb) that it's going to take some processing - it bears little resemblance to the data I currently fetch. I need a plan.

The Plan

Firstly, to avoid confusion, let's call the data I already had (and continue to fetch) as DATA-A, and this new data in its different form as DATA-B.

Having extracted DATA-B (and backed it up), I need to process data for days I already have in DATA-A. Let's call the resulting data, DATA-C. If DATA-C is the same as DATA-A, then my processing should replicate the data I'm missing.

Then for each sol that's missing from DATA-A, see if it exists in DATA-B and if so, process it and add the results to DATA-A. At the same time, I need to mark every record I'm adding as one I've processed myself, and therefore might not agree with NASA's equivalent data for that day. It should, but it's not guaranteed.

Then I need to highlight on the Mars Weather page, which data items in the tables I processed, and on the graphs (the rose graph and the Temperature Over Time graph, which elements were processed by me. This is not as straightforward as it sounds, especially with the TOT graph, but I'm writing a new graph script anyway to cope with data overflowing the graph boundaries, and the influx of all this new data will actually trigger that event early - so I can build it into the new graph rather than changing the existing one.

ToT Graph

Which brings me neatly to the Temperature over Time graph. At the time of writing this article, each data point occupies 1.3 pixels (width). As more data is added, that shrinks, until each data point is exactly one pixel wide. As new data is added beyond that date, each data point must occupy less and less of a pixel - what's known as sub-pixel metrics - which is not physically possible. A new way to represent the data must be employed.

There are many options for this, but I think the one I have chosen is a good one. I did have some weeks to go before the data became sub-pixel, but the addition of all this new data (nearly 100 sols will be added to the beginning of the dataset) will force me to process the new data and replace the graph at the same time. I'd better get cracking!

This website uses cookies. By using the website, you agree with storing cookies on your computer. Also you acknowledge that you have read and understand our Privacy Policy. If you do not agree leave the website.More information about cookies
blog/articles/info/weather_raw.txt · Last modified: 2020/11/07 10:12 by Phil Ide

Except where otherwise noted, content on this wiki is licensed under the following license: Copyright © Phil Ide
Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki
  • ISSplugin-autotooltip__default plugin-autotooltip_bigInternational Space Station

    This page has a portal through which you can view the current location of the International Space Station, updating in real-time. The tracker itself comes from the European Space Agency (ESA).

    There is also a handy link to get email notifications direct from NASA when the ISS is visible in your night sky.
  • Mars Weatherplugin-autotooltip__default plugin-autotooltip_bigMars Weather

    Weather Reports Insight Lander Perseverance Rover Curiosity data to come [Current positions of Earth and Mars] Image courtesy of and © Dominic Ford

    This image is updated once per week. A daily update is available on the Mars Solar Conjunctions page when such events are active.

    You can check the position of Mars vs the Sun and Earth for any date here:
  • Downloadsplugin-autotooltip__default plugin-autotooltip_bigDownloads

    Orbital Calculator Current Version This is a portable application, meaning there is no installer. Just unzip into a folder and run the executable. Since it is portable, it can be run from a USB stick.

    A full description and a list of the features are available
  • Orbital Calculatorplugin-autotooltip__default plugin-autotooltip_bigOrbital Calculator

    I needed to calculate orbital characteristics for any orbit around any gravitational mass (because I write sci-fi stories), so I wrote this program because I'm a programmer and I'm too lazy to do it by hand all the time. The program has expanded over the last few years, sometimes from suggestions by other writers. I make it freely available and as simple to use as possible. It now has a list of functions as long as your arm (assuming you're not a tentacle user), and the numbe…
  • Cookie Policy & GDPRplugin-autotooltip__default plugin-autotooltip_bigCookie Policy & GDPR

    There are NO tracking cookies on this site!

    The Cookie policy is straightforward. This site only uses cookies that are required to make your experience of using this site better, and only uses session cookies (see exceptions below). Session cookies are deleted when you close your browser.
  • Book Reviewsplugin-autotooltip__default plugin-autotooltip_bigBook Reviews

    Here you can find all the book reviews, broken down by genre, and further subdivided by author. If you want me to review your novel, please read my Rules on Reviewing.

    * Science-Fiction * Fantasy * Non-Fiction
  • Contactplugin-autotooltip__default plugin-autotooltip_bigContact

    If you have a burning desire to contact me, use the discussion form below. Anything you say remains private between you and me.