Cinnamon Hummingbird

Welcome to the Birder's Diary Forum for Support And General Questions

Use the Support forum for all questions or issues.
Use the Wish List forum to leave your ideas for improving Birder’s Diary.
Setup your Forum photo, profile, and signature here.

Sightings lost in t...
 
Notifications
Clear all

[Solved] Sightings lost in translation

78 Posts
3 Users
3 Reactions
8,918 Views
Posts: 72
Topic starter
(@graham-tebb)
Trusted Member
Joined: 5 years ago
[#1948]

Dear Jeff,

I really like the new "show only lifers" feature in the reports.  But mine has been giving me very funny results, reporting birds as lifers that I know I've seen many times before.  Of course, the program does not lie and when I looked for previous sightings of the species in question, there were none.  My assumption is that they have been somehow lost during the many taxonomic upgrades I have made over the years.

Is there any way to search a complete database (mine is around 200,000 records, so it would be a bore to do it by hand) for sightings that are not recognized in a given taxonomy?  That way we could easily pinpoint those that are "all dressed up with nowhere to go" - (c) Al Bondarenko a long time ago.  The reconcile wizard does not solve the problem:  I have removed most of the old taxonomies from the system (to save time and space) and have reconciled everything with the two version of IOC I still have.  But my fear is that there are loads of birds I've seen that are flying under the radar.  Any thoughts?

All the best,

Graham

 


Reply
77 Replies
Jeff
Posts: 1440
 Jeff
Admin
(@jeff)
Creator & Technical Support
Joined: 6 years ago

Hi Graham,

I just spent 45 minutes writing an in-depth analysis of what happens and how to resolve this kind of thing; not just for your case but for others that might find themselves in this situation. And then the browser bombed and I lost it all. After a few expletives, you are now getting the much sorter version.

Yes, Features | Sightings | Orphaned Sightings is the tool you want. 

  • Birder's Diary NEVER deletes Sightings unless the user explicitly deletes and confirms deletion.
  • Your missing Sightings are likely due to incomplete Reconciles against all older taxonomic lists. Since you have removed most of your older lists, the Orphaned Sightings feature is the way to find them.

Use the Orphaned Sightings feature, choose the Ignore Reconciled Sightings option and leave everything else blank. Click Search.

Now you can edit any Orphaned Sightings and assign them to the proper Avilist Thing.

Please update us on what you find and let me know if you have any other questions.

Happy Holidays.


Reply
(@graham-tebb)
Joined: 5 years ago

Trusted Member
Posts: 72

Dear Jeff,

What a wonderful feature!  I must have overlooked it one of the many upgrades; it seems massively useful:  thanks very much!

On the down side, it doesn't solve my problem and the sightings I was missing are still not showing.  I have noticed two "missing" species:

Madagascar Cuckooshrike - no problem.  This one was "automatically" changed to Comoros Cuckooshrike, which I have never seen.  I've changed all my sightings to Mad CS and all is well.

Cobalt-rumped Parrotlet - not picked up by the tool, which I guess means it is recognized by one of the IOC taxonomies I still have installed.  I'll try reconciling with them both (I've already done the latest one but maybe something went wrong when I upgraded from the previous version of the tax list) and report but it is puzzling that, e.g., Thing 24206 is not picked up at all by AviList.  This refers in IOC to the flavissumus ssp of Cobalt-rumped Parrotlet; the xanthopterygius ssp has also gone missing in action.  Very puzzling.

The worrying thing is that these two species were the ones I happened to notice (unexpected "lifers" on trip reports of species I knew I had seen many times before).  I can only guess how many gremlins are lurking in the dark corners of my database ...

All the best,

Graham


Reply
Jeff
Posts: 1440
 Jeff
Admin
(@jeff)
Creator & Technical Support
Joined: 6 years ago

Hello Graham,

Sorry for my tardy reply. Things have been hectic here. Playing catchup.

I am assuming that you found this first species in the Orphaned Sightings feature.

Posted by: @graham-tebb
↑

Madagascar Cuckooshrike - no problem.  This one was "automatically" changed to Comoros Cuckooshrike, which I have never seen.  I've changed all my sightings to Mad CS and all is well.

The IOC 2024 Madagascar Cuckooshrike did not change in AviList 2025. They both share the same underlying things. As per the IOC2024-to-Avilist2025.csv file that I shipped with the Avilist installer and can be found in "C:\Birder's Diary\Taxonomy Downloads" directory. So, this orphaned sighting must have been from years past. And, as you have noted, simply edit that sighting and change it. Check that one off.

Posted by: @graham-tebb
↑

Cobalt-rumped Parrotlet - not picked up by the tool, which I guess means it is recognized by one of the IOC taxonomies I still have installed.  I'll try reconciling with them both (I've already done the latest one but maybe something went wrong when I upgraded from the previous version of the tax list) and report but it is puzzling that, e.g., Thing 24206 is not picked up at all by AviList.  This refers in IOC to the flavissumus ssp of Cobalt-rumped Parrotlet; the xanthopterygius ssp has also gone missing in action.  Very puzzling.

Not sure I am following this here Graham. Where are you getting the ThingId 24206 from?

Why is this bird a problem? Is it that you just noticed that it is not on your Life List but it should be? Then, if that Sighting exists, it is attached to one of the taxonomic lists in your database - if the Orphaned Sightings feature did not pick it up.

Using Avilist 2025 as the newer taxonomy in the Reconcile Wizard, work backwards specifying the latest IOC tax list; click Analyze and see if it shows up. If not, move back to the IOC 2023, and so on.  Let me know on this please.

Posted by: @graham-tebb
↑

The worrying thing is that these two species were the ones I happened to notice (unexpected "lifers" on trip reports of species I knew I had seen many times before).  I can only guess how many gremlins are lurking in the dark corners of my database ...

This would be a first time if that is happening. 

I would like to see your database so I can follow along with what you are seeing. If you don't mind, when you get a moment, ZIP/compress your "C:\Birder's Diary\data\database.mdb" database file and upload it to me using the Support >> File Upload page on this site.

Thanks. 

p.s. heading out for a 4 day birding adventure. Back on Sunday.


Reply
Posts: 72
Topic starter
(@graham-tebb)
Trusted Member
Joined: 5 years ago

Dear Jeff,

Four days off?  And for an "adventure"?  When did you last do that?  I hope it went well - did you finally nail Aztec Thrush?

Okay, some background.  I have been using BD since the 90s and have had dozens of taxonomic lists.  A couple of years ago I deleted most of them and I now have only IOC13.2, IOC14.2 and AviList installed.  The Orphaned Sightings feature finds nothing (nothing at all; I have been very careful with all the updates) and the reconcile wizard reports that all my sightings have been reconciled from IOC13.2 to AviList and from IOC14.2 to AviList.  As far as the program can see, then, my database is fine.

A simple query using the IOC14.2 taxonomy tells me that I recorded 16 species at a particular site in Brazil in 2007 (see screenshot).  When I alter the query to use AviList (changing nothing else, I promise), the number drops to 15 species and Cobalt-rumped Parrotlet is missing (second screenshot).

ScreenHunter IOC142

 

ScreenHunter AviList

There is clearly something wrong with the way the system treats Cobalt-rumped Parrotlet.  A check of all my sightings of the species with AviList only finds sightings in 2024 and 2025, while the same query with IOC14.2 confirms that I also saw it in 2002 and 2007 (two more screenshots; query identical apart from choice of taxonomy).

ScreenHunter 001
ScreenHunter 002

Perhaps the problem is peculiar to Cobalt-rumped Parrotlet?  To check, I did the same thing with Mad Cuckooshrike (as the program had altered my earlier sightings to Comoro Cuckooshrike in one of the countless reconciliations I have performed - I edited them to Mad Cuckooshrike).  The results were slightly different:  the query found all my sightings of the dry-country form (ssp. pallidus) with AviList but ignored the other subspecies (nominate), although this was reported correctly when I used the IOC14.2 list (simple query by species, nothing else changed).

ScreenHunter 004

 

ScreenHunter 003

There is clearly something horribly wrong with the way my sightings are being recalled when I use AviList but I cannot say whether the problem is in the database or in the program.  I'll send you my compressed database so you can check when you have time.  Maybe updating to AviList will end up costing me fewer than 86 species - perhaps some of them are simply not being reported!

Sorry to cause you all these problems so close to Christmas ...

All the best,

Graham


Reply
Posts: 72
Topic starter
(@graham-tebb)
Trusted Member
Joined: 5 years ago

P.S.  I can't upload my database for you as even the zipped version is 206 KB, which apparently exceeds the limit.


Reply
Jeff
Posts: 1440
 Jeff
Admin
(@jeff)
Creator & Technical Support
Joined: 6 years ago

Hi Graham,

Thanks for the details. I am back now. No bird lifers. Maybe next time. Did get a few plant and bug lifers.

DATABASE UPLOAD

I will need your database. What is the size of your current un-zipped database? C:\Birder's Diary\data\database.mdb? When you Zip it, it will be smaller. There should be no size limit on the upload. 206KB would not be correct? Perhaps you meant 206MB. It will not allow you to upload the MDB file, only a ZIP file. Please try again, give me the details on both files and what the upload says exactly. Screenshots if possible.

COBALT-RUMPED PARROTLET viridissimus

I am going to keep this simple now and focus solely on this Sighting.

When you Reconciled from IOC 2024, your sighting of the subspecies F. x. flavissimus would have needed to have been reconciled from IOC to AviList. I would have needed to have been reconciled from F. x. flavissimus to F. x. flavissimus - the same subspecies. Sounds counterintuitive, but it is complicated and has to do with Clements, IOC and other taxonomic lists being combined into AviList. You should have made that move.

Here is what I need and what I am missing and should tell the story:

  1. You give me screenshots of the View/Edit Sightings window, but not the Setup. I need the View Sightings Setup screenshots that produced those list of sightings. In each case.
  2. Then, 
    1. Open Features | Advanced | Reconcile Sightings...
    2. Enter IOC 2024 and AviList 205 and click Analyze...
    3. Give me that screenshot.
    4. Now, click the report button, Zip and email (support@BirdersDiary.com) me that HTML report file that is created: C:\Birder's Diary\ReconcileReport.html
    5. This will tell me what you did with that IOC 2024 F. x. flavissimus sighting.
  3. And finally, ZIP/compress and send me your database.mdb file, if #1 and #2 above haven't already resolved the question.

I haven't found any glaring holes/errors yet in the Reconcile from IOC/Clements to AviList; and hopefully this will provide the story to explain what you are seeing. 

Let's start here.

Thanks much - looking forward to this information.


Reply
Posts: 72
Topic starter
(@graham-tebb)
Trusted Member
Joined: 5 years ago

Dear Jeff,

Sorry you came back empty-handed.  That Thrush must be out there somewhere, waiting for you!

Thanks for replying so soon.  To answer the simple questions first:  my database file is 1,329,920 KB and the zip version is 211.5 MB.  I have tried to upload this to your system, both in the office and at home.  In both cases I received the message, "File Graham-Tebb-database.zip not unloaded. Upload failed! The uploaded file exceeds the file size limit."  I am attaching a screenshot to prove I am not dreaming.  The problem with sending you my database remains.

Screenshot file not uploaded

 

I'll follow steps 1 and 2 in your instructions right away.  Stand by for some fun!

All the best,

Graham


Reply
Jeff
Posts: 1440
 Jeff
Admin
(@jeff)
Creator & Technical Support
Joined: 6 years ago

Hi Graham,

Ok. Do this to see if we can get the ZIP file down to a smaller size.

  1. Perform a backup (File | Backup/Restore...) of your database.
  2. Check the size of your database.mdb file now to confirm that it is smaller than before.
  3. Now Zip/Compress this MDB file and see what the new size of your ZIP file is.
  4. Now try uploading that new ZIP file.

Thanks.


Reply
(@graham-tebb)
Joined: 5 years ago

Trusted Member
Posts: 72

@Jeff Thanks - I should have thought of that trick!  You should now have my database; the new version was only 53 GB and uploaded without a hitch.  I'm afraid the file is simply named "database.mdb.zip" but you'll find it.

By the way, the screenshots of the View/Edit Sightings I sent you weren't exactly the same as the searches I used to produce the results I sent a few days ago.  They give the same sightings but in a different order to the original ones.  I promise you that the anomalies are still there!

All the best,

Graham


Reply
Posts: 72
Topic starter
(@graham-tebb)
Trusted Member
Joined: 5 years ago

Dear Jeff,

Here you are, then. Screenshots of the reconciliation between IOC14.2 and AviList - I'll upload the reconciliation report immediately.

All reconciled

And here are two fairly tedious screenshots of the View/Edit Sightings that I used to generate the lists of Cobalt-rumped Parrotlets I sent you last time.

AviList search
IOC search

Not desperately exciting - I have run thousands of searches of this kind in the past.  But why do the two give me different results?  The reconcile appears to have worked perfectly.  What am I doing wrong?

Thanks again for the help, and all the best,

Graham


Reply
Posts: 72
Topic starter
(@graham-tebb)
Trusted Member
Joined: 5 years ago

P.S.  I uploaded the reconcile report without any difficulty.  In case the problem with my database stemmed from the file name, I tried renaming it to "GTdatabase.zip".  The same error appeared - the file is too big.


Reply
Jeff
Posts: 1440
 Jeff
Admin
(@jeff)
Creator & Technical Support
Joined: 6 years ago

Hi Graham,

I have your Database now; so I can reproduce everything else. Thanks much!

I am focusing only on the Forpus xanthopterygius flavissimus sightings issue now. When we figure out what is going on here, that will likely be the answer in the other situations.

  • Your Reconcile Report confirms that you reconciled F. x. flavissimus sightings from IOC to AviList correctly.
  • When I search for all F. x. flavissimus sightings using AviList, I find 5 sightings.
  • When I search using IOC 2024, I find 13 sightings!
  • So, 8 sightings are missing when using AviList. Odd, that the 5 from "Brazil, Ceara..." are the 5 that are found using AviList.
image

Ok - I am digging into this. I have been on this for over an hour now, and figured I ought to update you. There are some anomalies here. I will figure it out and update you soon. 

Hang tight.


Reply
Posts: 72
Topic starter
(@graham-tebb)
Trusted Member
Joined: 5 years ago

Dear Jeff,

Many, many thanks.  My biggest concern was that you would not be able to reproduce the problem, in which case I would have guessed that something had gone wrong with updating my software to 6.3.  I'm glad that's not the case.

I suspect the difference between the first "Ceara" sightings and the others is related to when I entered them.  In each case, I dutifully typed everything in, using the current taxonomy, when I returned from the various trips, so was using different taxonomic lists all the time.  Did you change the ThingID of this beast at all on the way?

But the main reason to write now is to remind you that we have a huge time difference and I shall be logging off and going to bed soon.  Which means that you should not feel at all rushed; I can enter new sightings normally and most functions seem fine.  And sadly, the reconcile report confirms that I have lost out big-time on the conversion to AviList!

All the best,

Graham


Reply
Page 1 / 6
Share: