Importing data from Meteohub

Discussion about upload/download of data, import/export of foreign formats, etc.
Post Reply
oliverb
Posts: 38
Joined: Wed Mar 31, 2010 8:38 am
Location: Kenley Surey UK
Contact:

Importing data from Meteohub

Post by oliverb » Wed Mar 31, 2010 12:37 pm

I have data from my Meteohub going back to 2007. I have gone to "Inspect Data" from Meteoplug and selected "Import" I presume raw data has to be loaded month by month from the weather folder?
I have uploaded 2 months of data but this is not on Meteoplug.
Am I doing something wrong or is there a way of uploading all the data in one go?
Brett.

oliverb
Posts: 38
Joined: Wed Mar 31, 2010 8:38 am
Location: Kenley Surey UK
Contact:

Re: Importing data from Meteohub

Post by oliverb » Thu Apr 01, 2010 11:58 am

My data seems to be online now, must take some time to process.
Still not found a way to upload all my data so will do 1 month at a time from last month working backwards.

User avatar
admin
Site Admin
Posts: 332
Joined: Sun Feb 07, 2010 9:22 pm

Re: Importing data from Meteohub

Post by admin » Fri Apr 02, 2010 8:22 am

Hi,
unless you have a very fast internet connection I would recommend to zip, gzip or bzip2 the files before uploading. When upload is finished, Meteoplug server will queue a request to generate aggregated data for uploaded raw data. This needs to be finished to make use of data in graphs/tables etc.

You will find a UPDATEDB-message in your server log like this:
UPDATEDB 0E:AD:00:00:00:03: time 100s/30s [2010-01-01 00:00:00 - 2010-01-31 23:59:53]: ........

As your data that gets uploaded from Meteoplug client also generates UPDATEDB entries in server log, you will have to look for one with a update time period that fits to the data you uploaded.

I would not recommend to upload data of more than one month with one upload request, as web interface might run into a time out when upload and raw data import is taking too much time.

skyewright
Posts: 69
Joined: Tue Mar 16, 2010 12:05 pm
Location: Isle of Skye
Contact:

Re: Importing data from Meteohub

Post by skyewright » Fri Apr 02, 2010 10:14 am

oliverb wrote:I have data from my Meteohub going back to 2007.
I have successfully imported my full archive, going back to 2008-03. Using compressed archives, one month at a time, did the trick for me. On a couple of occasions the data for a particular month didn't appeared after the process seemed to be complete, but that was solved by a re-import. I presume that those problems were down to a timeout on my fairly slow connection (we are lucky to have Broadband at all here! :D ).
David

oliverb
Posts: 38
Joined: Wed Mar 31, 2010 8:38 am
Location: Kenley Surey UK
Contact:

Re: Importing data from Meteohub

Post by oliverb » Fri Apr 02, 2010 10:32 pm

Thanks for the replies. I now upload a months data every now and then and allow meteoplug time to process it . A quick look at the export select box on inspect data confirms what is already held on meteoplug.

User avatar
admin
Site Admin
Posts: 332
Joined: Sun Feb 07, 2010 9:22 pm

Re: Importing data from Meteohub

Post by admin » Fri Apr 02, 2010 11:27 pm

yes, this shows available raw data... but aggregated data will take a while because quite a queue has to be handled before your request comes through.

gibsonmb
Posts: 12
Joined: Thu Apr 22, 2010 8:30 pm

Re: Importing data from Meteohub

Post by gibsonmb » Tue Jun 08, 2010 6:35 pm

I've tried this a few times now but it never seems to work. On the inspect data tab, I select browse at the and of the import data file box. I then select a raw file from the weather directory of meteohub fro the month I want to upload. I then press import. On the progress bar at the bottom of the webscreen I get "waiting for https://www.meteoplug.com/cgi-bin/meteoplug.cgi... as the green progress indicator moves across the bottom of the screen. Am I doing it right? It never seems to end as such and that's the end of that. The data is not imported. :(

User avatar
admin
Site Admin
Posts: 332
Joined: Sun Feb 07, 2010 9:22 pm

Re: Importing data from Meteohub

Post by admin » Tue Jun 08, 2010 10:31 pm

may be your upload is too slow. please try ziping the file before uploading. that should work for sure.
did you try another browser? what browser do you use?

Post Reply