Welcome to the Birder's Diary Forum for Support And General Questions
Use the Support forum for all questions or issues.
Use the Wish List forum to leave your ideas for improving Birder’s Diary.
Use the Community Sharing forum for sharing Photos, Trips, Stories, etc.
Setup your Forum photo and profile here.
I have just been importing multiple lists from eBird for the first time in BD 6.0. When I open the Import window, and Click the Import Sightings from eBird export file button, the program properly imports the first file, and allows me to delete it when finished. Then, when I try to import the next eBird file, after navigating to and choosing the next file I want to import, I SOMETIMES get the following series of errors:
Then two of these
and finally
The first seems to imply that BD is looking for a default .csv file, which does not exist, and is not the file I have chosen.
After this error, the behavior of the program is again unpredictable. Sometimes, after this first failed import, when I reselect the same file choosing operation without closing the 'Import/Export' bundle of windows the program will properly read the file. Other times, it will again generate the same set of errors, and this can continue over many iterations. Reading the file might be successful after 2, 3, or 4 tries. After that, I have given up and closed the Import/Export window bundle, and this always seems to clear the error, and enable at least one more file to be read.
Another, possibly unrelated behavior is that after a succesful Import, the cursor can either return to its ready state, or sometimes remains as the Windows 'waiting' state of a blue circle. However, this cursor state does not seem to predict whether the next attempt to import will be successful.
The error occurs about a quarter of the imports I attempt, but again it might go 10 files without an error, then occur for two consecutive files - or not.
What makes this particularly annoying is a perhaps separate issue that the program forgets that I have told it to substitute 'Rock Dove' for the eBird 'Rock Dove (Feral Pigeon)', (as well as other rarer substitutions) so each time I exit the Import/Export bundle, the substitutions have to be reentered.
Hi David,
Thanks for all the details and the screen captures. Helps a lot.
[Previous text removed]
I had written some text thinking that this was due to having the file open in Excel. But after digging into some code, I find that this is not the case and already had some code to deal with this situation: RTE 3051. From v5.3.13, 4-Jan-2022. This is from a previous post from you David.
errRetryCnt starts out at 2. So you get two retries before it fails. This seemed to fix your problem from one year ago. Oddly, almost exactly the same time of year. I wonder if this has something to do with the first week of the year??? Weird, but can't rule out the coincidence.
After 2 tries, then you get the other errors.
More digging... see next two posts also please.
Here's the link to last year's post in these forums.
Ok - now I remember in full. You have a multi-threaded very fast computer.
What my code is doing, which should be sequential in ALL cases, is that it closes the CSV file and in the next statement attempts to import it into Access. But, in your case (no one else has ever reported this), the Close-File doesn't complete before the Import-File attempts to execute. This should never happen on any computer running Windows as far as I can see. But it does happen on your computer.
So, for that case, I bundled in a wait 1 second and try again on the import. Do this twice before quitting and reporting the error. And, last year, that seemed to solve the problem for you.
And here we are again almost exactly 1 year later. Queue the Twilight Zone music in the background.
Another question David.
Are you importing multiple files at once? You can do that. You can select 2 or more files in the file selection window to import, and they will be imported one after the other. Or, are you just selecting one file at a time?
Also, can you send your "eBirdSightingsImport.log" file to me at Jeff@BirdersDiary.com? Thx
@jeff I am importing lists one at a time, so I can have the pleasure of Petey telling me about each life bird. When I import multiple lists simultaneously, it tries to tell me ALL the life birds all at once - a different sort of error.
Yes, I do have amultithreaded computer, but it is not so fast these days. The error I am describing takes a few seconds to appear after I hit Import.
I trid to attach the Log file, but the forum site returns an error that the file type is not allowed.
Log file sent to your email address from above.
Hi David,
I did not receive the log file. But that is ok. I asked for that before I found the old problem from a year ago. I don't think the log file will contain any info in it.
I am not sure what to do with this at this point. It is definitely something that should never be happening on Windows. It is getting sequential code executed out of order. I suspect that Windows is queuing the request to close the import file to one thread and then continuing forward with the code that tries to use that file as an import target before the file has actually closed. This should never happen.
I have an extremely fast multi-threaded AMD Ryzen5 cpu, on one of my computers. And I can't get this to happen no matter how many times I try, on this or any of the other 4. I wrote code to automate and import the same file over and over as fast as it could 100X. No errors.
Last year I put in the sleep 1 sec and retry 2X before failing and that seemed to solve your problem until now. Do you only import eBird files this time of year each year; or do you do this all year long, but only have this problem at this time of year? I am just looking for some sort of pattern that I can take advantage of and try to work around this.
For your info, as maybe this triggers an idea for you, here is the process...
- Open the eBird CSV file you specify
- Change from eBird format to BD Sightings format. Read the eBird CSV file one line at a time and write it out to eBirdSightingsImport.csv in BD-Sightings-Import format.
- Close both the eBird and the BD CSV files.
- Now ask Access to import the eBirdSightingsImport.csv file just created into a temporary table in the database.
This is where the error occurs. Access attempts to open and read/import the newly created CSV file and finds that it is still open; and therefore fails. The Close() in step #3 has not yet executed for some reason.
This all works fine on your computer 75% of the time, but every 1 out of 4 (or 10, or 20) times, we get to step #4 without step #3 having completed. This is the part that should never happen. As if it did, code all around the world would break if Windows lets code operate out of order.
I love these kind of whodunnit type of mysteries. But not sure what step to take next to resolve this. Let's keep this open for a bit and see if I can't think of something. Please let me know if you come up with an idea.
I did a big batch of imports without errors in June 2022, but that was in BD5.3. I would look to differences in version number before import date.
Here is an idea: Perhaps Windows HAS executed the close command but the underlying hardware is busy with something else?
Can you directly query Windows as to the status of the file before moving to read it in? So instead of just waiting a set period after you write the file, test for the availability of the file, as in something like
if exist(filename) { read the file}
else {wait a while before testing existence again}
?
David,
I will release something along those lines for you to test later this week. I have a few ideas I want to test out before I release anything.
Will update here when ready.
Boy, oh boy! As much as I want to blame everything else for my bad code, it always comes back to my bad code! 😜
I found the problem. After writing a lengthy reply here on how this must be a bug in Windows as nothing in VB6 is multi-threaded, it always helps to write down your thinking just so you can see how wrong you are. There is one thing (not in my code, but in the VB runtime engine) that is multithreaded! The VB Runtime Garbage Collector. Then, I thought, well, what could I be doing such that the GC might be exhibiting this behavior. Well, only if I got rid of the Class Object before closing the file. So I went looking. And sure enough, I did not close the file; simply deleted the Class Object. Which does the same thing - e.g. Close the File - but only when the GC gets around to it; i.e. asynchronously. And so, once in a while, the GC hasn't gotten around to doing this before the Access Import statement executes!!! That is why it works 80% of the time, and fails the other 20%, or so in your case.
Now, I am unable to test this, as I am unable to reproduce this on my development machines. But that is also explained by the Garbage Collector. Because in VB development mode, everything is STRICTLY SEQUENTIAL, even the GC!!! So I was never going to put all of these pieces of the puzzle together without this trial-n-error process that I had to go through, in order to come around to this Eureka moment.
So, while I am not 100% certain that this is the fix, it is consistent with All of the symptoms so far experienced, and is the only answer that comes close.
I will have Build 23 up in a few moments for you to download and try. Please get back to me after testing and let me know.
Build 23 is now available!
Going to go ahead and mark this solved David. When you get around to testing this, please let me know. Thanks.
