Page 1 of 1

Importing data from Meteohub

Posted: Wed Mar 31, 2010 12:37 pm
by oliverb
I have data from my Meteohub going back to 2007. I have gone to "Inspect Data" from Meteoplug and selected "Import" I presume raw data has to be loaded month by month from the weather folder?
I have uploaded 2 months of data but this is not on Meteoplug.
Am I doing something wrong or is there a way of uploading all the data in one go?
Brett.

Re: Importing data from Meteohub

Posted: Thu Apr 01, 2010 11:58 am
by oliverb
My data seems to be online now, must take some time to process.
Still not found a way to upload all my data so will do 1 month at a time from last month working backwards.

Re: Importing data from Meteohub

Posted: Fri Apr 02, 2010 8:22 am
by admin
Hi,
unless you have a very fast internet connection I would recommend to zip, gzip or bzip2 the files before uploading. When upload is finished, Meteoplug server will queue a request to generate aggregated data for uploaded raw data. This needs to be finished to make use of data in graphs/tables etc.

You will find a UPDATEDB-message in your server log like this:
UPDATEDB 0E:AD:00:00:00:03: time 100s/30s [2010-01-01 00:00:00 - 2010-01-31 23:59:53]: ........

As your data that gets uploaded from Meteoplug client also generates UPDATEDB entries in server log, you will have to look for one with a update time period that fits to the data you uploaded.

I would not recommend to upload data of more than one month with one upload request, as web interface might run into a time out when upload and raw data import is taking too much time.

Re: Importing data from Meteohub

Posted: Fri Apr 02, 2010 10:14 am
by skyewright
oliverb wrote:I have data from my Meteohub going back to 2007.
I have successfully imported my full archive, going back to 2008-03. Using compressed archives, one month at a time, did the trick for me. On a couple of occasions the data for a particular month didn't appeared after the process seemed to be complete, but that was solved by a re-import. I presume that those problems were down to a timeout on my fairly slow connection (we are lucky to have Broadband at all here! :D ).

Re: Importing data from Meteohub

Posted: Fri Apr 02, 2010 10:32 pm
by oliverb
Thanks for the replies. I now upload a months data every now and then and allow meteoplug time to process it . A quick look at the export select box on inspect data confirms what is already held on meteoplug.

Re: Importing data from Meteohub

Posted: Fri Apr 02, 2010 11:27 pm
by admin
yes, this shows available raw data... but aggregated data will take a while because quite a queue has to be handled before your request comes through.

Re: Importing data from Meteohub

Posted: Tue Jun 08, 2010 6:35 pm
by gibsonmb
I've tried this a few times now but it never seems to work. On the inspect data tab, I select browse at the and of the import data file box. I then select a raw file from the weather directory of meteohub fro the month I want to upload. I then press import. On the progress bar at the bottom of the webscreen I get "waiting for https://www.meteoplug.com/cgi-bin/meteoplug.cgi... as the green progress indicator moves across the bottom of the screen. Am I doing it right? It never seems to end as such and that's the end of that. The data is not imported. :(

Re: Importing data from Meteohub

Posted: Tue Jun 08, 2010 10:31 pm
by admin
may be your upload is too slow. please try ziping the file before uploading. that should work for sure.
did you try another browser? what browser do you use?