SID detection with UKRAA VLF radio

[HN50]

3 hours ago, Tomatobro said:

I am fortunate that I have access to an expensive fluke multimeter and a frequency generator.

I will post some screen shots later on. I think that Spectrum lab offered a range of "standard" vlf frequencies and 23.4 is the only one I am currently logging ( and noise floor level) as all the others either give no signal or are seriously affected by local interference such as our induction hob and gas boiler.

I am reasonably confident that I am recording DHO38 because rotating the antenna gives me a peak signal when pointing towards the station in Germany and I am seeing the early morning maintenance shutdown sessions. I can see a problem trying to extract small signal changes from the data as the noise span covers some 3db (at least). I am taking 1 reading every 30 seconds as a plot. Perhaps averaging over (say) 5 readings might help but I have not looked at this yet.

Hopefully my signal maintenance dips will match up with your UKRAA receiver data

Regards

 

Hi,

I could quite happily own a Fluke multimeter..!
I posted my data for the last week so would be interested to see how it compares, not least as there were some strange readings yesterday morning around the maintenance window.

Dave

[Tomatobro]

Not had chance to do anything with this yet but will compare tomorrow.

[Tomatobro]

 

The shutdown looks similar to yours

 

[HN50]

21 hours ago, Tomatobro said:

 

The shutdown looks similar to yours

 

Hi,

Thanks for posting your output, that does look rather similar.  When I first saw the 02/05/2020 plot it did make me wonder if there was a problem with my setup but it appears that it is a real result.

Dave

[Tomatobro]

At least we wont be tearing the kit apart trying to find a fault........

I am now looking at ways to semi automate the extraction of meaningful data from the spectrum lab text file.  I still think the UKRAA receiver is the way to go but will give it one last try. Will post the results when the work is done.

[Tomatobro]

Spectrum lab's txt file is quite extensive so I copy it into Libre office calc, select the column of data of interest and paste it into a new spreadsheet. I then strip out the decimal and export as a text file.

Using TeraTerm I send this file to my hardware that writes it to an EEPROM and then read it back to MakerPlot doing any data processing necessary.

I take a snapshot of the MakerPlot graph for later reference.

The hardware will allow direct recording so if I build a UKRAA radio I can dispense with the laptop . I will have to rebuild the hardware using a more powerful processor as the one I am using at the moment has 1k of memory and its about all used in the above process.

Sample Graph attached

Edited May 9, 2020 by Tomatobro
error

[HN50]

On 09/05/2020 at 13:42, Tomatobro said:

Spectrum lab's txt file is quite extensive so I copy it into Libre office calc, select the column of data of interest and paste it into a new spreadsheet. I then strip out the decimal and export as a text file.

Using TeraTerm I send this file to my hardware that writes it to an EEPROM and then read it back to MakerPlot doing any data processing necessary.

I take a snapshot of the MakerPlot graph for later reference.

The hardware will allow direct recording so if I build a UKRAA radio I can dispense with the laptop . I will have to rebuild the hardware using a more powerful processor as the one I am using at the moment has 1k of memory and its about all used in the above process.

Sample Graph attached

 

Hi,

That is interesting, if you get rid of the laptop what is the unit a custom built computer/microcontroller?  I ask that as I see you mention EEPROM which is available on the Arduino.  You only get 100k (I think) writes Arduino though which would get used pretty quick with the loop that is triggered every minute.  How many writes to do get before it corrupts?

Well, here is the Sunday post.

As I ever I resisted the temptation to hoick out the card before today.  I copied over the data into LibreOffice Calc and analyzed it. 

The good;

>>> Orienting the antenna did not seem to make too much difference.  I realised that I was already nearly pointing in the right direction.

>>> I don't think the significantly higher temperatures this week affected the radio

The bad (well, not sure, I don't think I have any control over it but it becomes clear from the plots).  There is a time jump in the first screenshot as I had it switched off for a few hours doing work on it.

'Something' happened on the 04/05/2020.  Prior to that the signal strength of the transmitter was of a range I had seen before, however after the maintenance window on Star Wars Day (May 4th, boom boom) the signal strength of the transmitter has been significantly down all week up to and including today.

I have not made changes to the radio's tuning and the fact that this behaviour takes place after the switch on on the 4th makes me wonder if the transmitter is running at a much lower power..?  I suppose I will get an idea when I pull out the data next Sunday.  So I won't worry too much at this stage as my setup still seems to be working.

GIF for the week.

I have also started looking at the Arduino MKR1010, I have played around with some of the sketches and have got it logging details of WiFi connections available around my house.  I think there will be a bit of a learning curve with it as I have not quite been able to get some of the WiFi enabled projects working, though they do focus on it being used as a server and as a data logger I think it would be a client.

In addition there will be some rewriting as it has its own RTC built in and I am not sure if the DS3231 is compatible (the library for this is filed under 'Incompatible' in the Arduino IDE when you select MKR1010).  Plus it runs on 3.3V not 5V so the chassis I plug it in to will need to be quite different to the YABBA ones I have built, but I had expected that. 

The biggest problem I have is that after installing the MKR1010 software I don't seem to be able to program my YABBA boards any more, I get a program error when I upload to the board(!!!).  I was going to reset the RTC to be on GMT (I realise mine is set to BST) today but at the moment I can't.  Let's hope my setup is reliable as I can't currently make changes..!

Anyway, let's see how things go.  If I can't program my existing boards then I have to migrate fully to MKR1010 and done with.  I guess I had better get investigating it...

Thanks for reading.

[Tomatobro]

The EEPROM onboard the chip has a limitation as to the number of read writes so I don't use it. I write the data to a 24lc256 EEPROM. There is a 512 version which is unused on the UKRAA receiver and I intend to  use this .

With EEPROM's its good practice to write 255 into each byte and then read all the locations back to check that all cells are working. This is the erase process that I use to give me confidence that the storage chip is good. Also I read in a word variable and check that if it equals 65535 then my data read has come to the end of the sequence.

I am using a PIC 16f84 for my initial tests and its limited to 1k of memory so its a tight fit to get all the routines squeezed in.

[Tomatobro]

Here is a picture of my data capture board.

Tomato donated a couple of stair newels which I used to make the antenna frame and everything so far has involved a zero outlay. The financial director has asked me not to put any new capex for astro items till the end of the month so will use what I have till then. Its also true to say that without the stimulation of your posts on the subject I would not have got this far. Nothing like having something to compare my results with👍

Edited May 10, 2020 by Tomatobro
correction

[HN50]

Hi,

I like it, I am trying to visualize it - that is pretty small, isn't it?  Do you have a block diagram of how your setup will work?

Well, a bit of a mixed bag in terms of progress this week.  Firstly, the good, my data capture is going on well.  The overall signal strength seems down as compared to earlier weeks, however the results seem to be stable.

There do not seems to be have many maintenance shutoffs either.  Weekly animation is below.

Another week and has passed and my rig seems stable, the logger and radio are working, and I am getting a steady profile.  I am interested as to why the overall signal is weaker as compared to the late in April, so I wonder if the transmitter is operating at reduced power?  I also checked the GOES X-ray flux data for the last 7 days but nothing bigger than A class has happened.

I have also produced an average plot by hour of all data logged so far which is starting to show that certainly the mean is gradually settling down to resemble a quiet day as extreme data becomes less significant.

And now the bad. 

1) I still can't push code out to the barebones Arduinos I have built, I keep getting an 'avrdude not in sync error'.  This seems to be a little cryptic to work out what is going on.  I have;

> Bought a new usb to serial chip in case this was the cause - it isn't as I still get the same problem

> Run the logger under external power to see if it is working or the Atmega328p chip is fried.  It works and happily logs, so that isn't the cause.

> Reburnt bootloader to chip to see if that had corrupted.  I could do this successfully however I still can't upload my code to the chip.

So still a problem that I can't push out updates..!

2) After spending a lot of yesterday and today on the above, I have been experimenting with getting the MKR1010 working if I can't now upload code to the Atmega328p models.  I have it writing to an SD card and have been trying to get it working with an RTC.  There is one built into the MKR1010 however it needs to be set via a WiFi connection and has no battery backup.  In initial experiments I have found that if you have a sketch that sets the clock via the internet and then upload the logger sketch the RTC starts up at the last time the previous sketch ran and does not increment between.  There may be settings that deal with this, I am still investigating how it works but I do not want it to be constantly connected to the internet in case I need this to be battery powered.  I am also seeing if I can use the DS3231 RTC as I like it being battery backed up and has several alarms built in to it.  So while I have not got round to building anything I am at least investigating what the MKR 1010 can do.

That all sounds good, the bad part in the above is in the last hour my laptop has started refusing to detect the MKR1010 at all.  I think the drivers became corrupted so now I can't upload to code to the new Arduino either. 

Three steps forward and three back this week, those emoticons are rather apt.

Thanks for reading.

 

Edited May 17, 2020 by HN50
Punctuation

[HN50]

Double clicking the reset button when connecting the MKR1010 to the laptop via USB seems to help the drivers get set correctly.

Beats me but there we are. 

Edited May 17, 2020 by HN50
Punctuation

[Tomatobro]

Ahh the delights of programming .............

I am in the process of building the Mk2 data logger computer. It will have a processor with 8k of memory (luxury!). The 15v power supply has arrived so slowly getting ready for the UKRAA receiver kit to arrive.

I have dabbled with ardunio boards but its been mainly about modifying scripts that other folks have written so I would not be much help with your issues I am afraid. I tend to stick with what I know well but can see the benefits of your approach as your data processing system is far more powerful.

I will post some detail and picture when it is closer to being finished.

 

 

Not much activity from the Sun........

[Tomatobro]

This is a recording of a few hours worth of data taken today using my UKRAA receiver. It seemed to make sense for a while but not from 15:40 onwards.  It looks normal enough at the start then it seems they had a problem and shut it down for maintenance. At the restart it looks normal then it was reduced to half power then shut down again and so on.  Looking at your 05/07 graph I can see a similar pattern

 

Edited May 22, 2020 by Tomatobro
update

[HN50]

1 hour ago, Tomatobro said:

This is a recording of a few hours worth of data taken today using my UKRAA receiver. It seemed to make sense for a while but not from 15:40 onwards.  It looks normal enough at the start then it seems they had a problem and shut it down for maintenance. At the restart it looks normal then it was reduced to half power then shut down again and so on.  Looking at your 05/07 graph I can see a similar pattern

 

Hi,

I will try and get the data out the SD card tomorrow, I am hoping to see a similar pattern.
Dave

[HN50]

Weekly update...

I pulled out the data and passed it through my trusty LibreOffice spreadsheet and generated the following GIF (I won't post each individual day but will do if asked).

The data looks okay and the unit is running as I would have hoped, but yes, quite an uneventful week.  If we are at a deep solar minimum then I guess we have a few years to get this all perfected!  I also recalculated the all time averages as well and has hoped the averages are stabilising with continued normal functioning.

@Tomatobro - I looked at 22/05/2020 to compare to what you had recorded.  The whole day looks like;

I see a drop at about 15:00 (my RTC is running on BST at the moment), so I zoomed in further.

I must admit I am not sure I see the same output, there is a dip in signal but I don't see that high/low output you were getting.  Do you have a plot for 23/05/2020 (mine is pasted below)?  There seems to be several on/off events in the evening so would be good to see if you detected these.

In terms of development, I have been looking at getting a MKR 1010 prototype running on a breadboard. 

I have got it writing to the SD card but one thing I have found is that the DS3232.h library I had been using is not compatible with the MKR 1010, I think there are some reserved keywords in there that clash and error when you compile.  I found this posting https://forum.arduino.cc/index.php?topic=626051.0 which refers to a particular v 1.0.3 version of an alternative RTC library to use so I downloaded that.  The keywords are different so I have spent today getting to grips with setting the time and using the alarm functions.

Pretty poor photo but that is just a sketch that shows the time and various alarm settings. 

So I think the next step will be to get the various bits (RTC/SD card/low power) all working together on the breadboard prototype.  I also need to check the voltage divider as I am not sure there is a setting to put the internal voltage reference to 1.1V in the MKR 1010 so the divider will need to drop the radio 5V to 3.3V instead.  I am not too concerned as I can set the ADC to 12 bit.

Anyway, things to do in the coming days.

Thanks for reading.

 

[Tomatobro]

Yesterday was the first real test of my newly constructed UKRAA receiver. Its clearly not working properly  based on yesterday's and today's log so will pull it out and take another look at it.

[HN50]

Just now, Tomatobro said:

Yesterday was the first real test of my newly constructed UKRAA receiver. Its clearly not working properly  based on yesterday's and today's log so will pull it out and take another look at it.

Sorry to hear that, hope you get it working.

[Tomatobro]

I have found that feeding the receiver with the right signal the outputs are correct. I based my antenna design on a  website that on reflection seems to be specific to using the sound card rather than the UKRAA receiver. I have ordered some 24 awg wire and will rewind the antenna.

The antenna as it stands responds well to the noise generated by the notebook screen. Those bumps in the graph are the notebook waking and turning on the screen when I go look for the current signal data.

[HN50]

1 hour ago, Tomatobro said:

I have found that feeding the receiver with the right signal the outputs are correct. I based my antenna design on a  website that on reflection seems to be specific to using the sound card rather than the UKRAA receiver. I have ordered some 24 awg wire and will rewind the antenna.

The antenna as it stands responds well to the noise generated by the notebook screen. Those bumps in the graph are the notebook waking and turning on the screen when I go look for the current signal data.

Hi,

Glad it looks like the radio is okay.

I am in the midst of 'Arduino Can't Find Drivers Part 746'.  I think though that I might have got to the bottom of it, I am writing it here as much for my reference as anyone else's! 

1. When CHRG light stars flashing and IDE can't find COM port, unplug Arduino MKR 1010
2. Restart laptop
3. Unplug I2C modules (in this case the RTC), also unplugged the SD card even though not I2C (I found a posting in https://forum.arduino.cc/index.php?topic=567417.0 that gave me an idea)
4. When laptop restarted plug in Arduino
5. CHRG light may flash but should hopefully go out after a couple of seconds, in Device Manager should see Arduino MKR WiFi 1010 under a COM port
6. Try uploading a sketch
7. Plug in I2C RTC and SD card
8. Should now upload code successfully in IDE
9. Universe is in balance again

That is a rather long way of saying turn it off and on again.  I like Arduino a lot but I find I can burn up hours trying to resolve these issues.

And now on to what I actually wanted to do today...

!

Edited May 25, 2020 by HN50
Words wrong

[Tomatobro]

 I missed the flare due to non availability of the kit. It will be interesting to see if you captured it

[HN50]

6 hours ago, Tomatobro said:

 I missed the flare due to non availability of the kit. It will be interesting to see if you captured it

!!!

Thanks for the heads up.

Things are looking quite lively still, I will leave it until late tonight or tomorrow and pull out the data.  

Fingers crossed...

Dave 

[HN50]

12 hours ago, Tomatobro said:

 I missed the flare due to non availability of the kit. It will be interesting to see if you captured it

Hi,

I will analyse properly tomorrow (I took some screenshots of the GOES data so will tie up times), but at first glance it does look like there has been some activity detected today.  My times are BST rather than UTC so I think the peak at 07:00 UTC has fallen into the 08:00 - 09:00 maintenance window, and I think the downward spike was the transmitter going offline.  However the GOES spike at 11:45 UTC corresponds with a sudden drop in signal in my plot at about 10:45 BST. 

So I think I have detected a flare.  

I will look again later when my brain is more awake as there are a few features to correlate, and it would help if I converted my times to UTC....    Interestingly they are negative which suggests to me that the changes in the ionosphere make the ground wave and skywave further out of phase (?).

Dave

[Tomatobro]

Nice plot. Very inconsiderate of them to switch off when a SID occurs! 

I am waiting for a UKRAA antenna tuning unit to calibrate my new antenna. This one has been made according to the instructions in the construction PDF.  If I cannot wait I will go into the "stores" and see if I can find some temporary capacitors to get it up and running.

[HN50]

1 hour ago, Tomatobro said:

Nice plot. Very inconsiderate of them to switch off when a SID occurs! 

I am waiting for a UKRAA antenna tuning unit to calibrate my new antenna. This one has been made according to the instructions in the construction PDF.  If I cannot wait I will go into the "stores" and see if I can find some temporary capacitors to get it up and running.

Thanks!  Doing battle with LibreOffice at moment to brink them together.  Hopefully will have a chart out soon.

[HN50]

2 hours ago, Tomatobro said:

Nice plot. Very inconsiderate of them to switch off when a SID occurs! 

I am waiting for a UKRAA antenna tuning unit to calibrate my new antenna. This one has been made according to the instructions in the construction PDF.  If I cannot wait I will go into the "stores" and see if I can find some temporary capacitors to get it up and running.

After a couple of hours of battle with LibreOffice Calc I have charted my data against the GOES x-ray flux data for the same time, and the results are below.

As I had thought the M-class (biggest since Oct 2020 https://www.swpc.noaa.gov/news/first-m-class-flare-observed-years) fell slap in the middle of the routine maintenance window and so was undetected.  Ain't that the way? 

However the following C-class flare did cause a SID as the signal strength for DH038 dropped appreciably and gradually recovered.  The data from both datasets correlates spot on. 

Throughout the afternoon and evening there were further events but I can't correlate them in the same way so I guess these events led to the ionosphere being disturbed and causing the signal from DHO38 to jump about.

So whilst I missed the big fish I did indeed detect a following flare.  I am very pleased, we can turn the internet off now.

I will have a think this afternoon about what I do next whilst I enjoy the sun, data attached if it helps anyone.

Thanks for reading. 

Dave

VLF_T2_20200530.TXT