Monday, February 18, 2019

The World's Premier Listing Software
for Birders and Naturalists!

Celebrating 24 Years Serving Birders
and Naturalists around the World!

Search this Site and the Forum Contents

Birder's Diary Forums

location in data imported from BirdBase
Last Post 02/17/2019 7:45 AM by Jeff Jones. 12 Replies.
Printer Friendly
  •  
  •  
  •  
  •  
  •  
Sort:
PrevPrev NextNext
You are not authorized to post a reply.
Author Messages
tapaculo
Yellow Warbler
Yellow Warbler
Posts:19


--
02/10/2019 4:32 PM

    I am an immigrant (or should I say refugee?) from BirdBase and need to deal with how location was dealt with differently in the data imported into Birder's Diary. Among other things,  changes are needed if I want to export my records into eBird and contribute to the "citizen science" aspect of that Website. Below is an explanation of the problem, and - since I couldn't find any discussion of this in the forums - a possible solution. I would be grateful for comments... or a better solution!

    In BirdBase (BB), there was a fixed list of locations, mostly countries and US states and a few world regions. All records were entered as part of a trip, which was associated with one of those locations and a single date. So that's where each record gets its location and date: from the trip with which it is associated, in effect its parent data. Unlike Birder's Diary (BD), BB had no hierarchy of trips. There was a fixed hierarchy between its states and countries and regions, for reporting purposes.

    Other information could be entered within the Description of the trip or the sighting note for the particular species. I chose to put further detail on the location into what seemed like the logical place: the Description for the trip.

    In importing records from BB, BD made sensible decisions, maybe the only viable options: it takes the Location and Date for each sighting from the respective fields in the associated BB trip... and also imports the trip and keeps each record associated with that Trip, and imports the BB Sighting Notes into the Comments of the sighting. BD does not appear to associate Trips with Locations; it allows for the beginning and end date of a trip, but the ones imported from BB will have the same date for each.

    Ideally, each record should be associated with a specific location, which may mean a birding site or even lat/long coordinates. So how do we get to that from the location that has been imported from BB into BD, with few exceptions simply a country or US state?

    Here is a possible solution:

    1. Create a new Location corresponding to each BB trip.
    2. Make each location a "child" of the default BD locations/countries. (This is essential.)
    3. Using View/Edit Sightings, display all the records for each trip.
    4. Marking all the records, use the Edit Selected Rows function to do a "mass edit" and change Location to the corresponding new Location. 

    Does that seem reasonable? Is there any way to accomplish this more efficiently, not doing it one BB trip at a time?

    p.s. An additional step that is not essential is to organize the trips. Note that the granularity of trips in BB is much finer than what I believe is the "best practice" in BD: if you travel to another country and spend some weeks birding there, that should be a single trip in BD. In BB, since you couldn't add locations, it seemed the best practice was to make each site a separate trip. So the solution for the trips imported from BB would be to create in BD a Trip representing the "whole trip" to that country... and then make that the Parent Trip for all the individual imported trips.

    Tags: BirdBase
    Jeff Jones
    Great Gray Owl
    Great Gray Owl
    Posts:8297


    --
    02/11/2019 8:17 AM

    Hi Steve,

    Thanks for posting. 

    Your assumptions above are correct. The import into BD knows how to convert most of your BB 2/3-letter-code abbreviations for location into BD names; e.g. "sd" = "South Dakato". And so on. However, there are a few mistakes as BB did not adopt the latest ANSI codes for locations and some (e.g. "Russia Georgia" might get assigned to "USA Georgia"). However, basically, this is how the assignment is done. Not Lat/Long, but strictly based on the BB 2-letter location code. As that is all that BB exports.

    Each Sighting from BB is assigned to a location, date/time and trip in BD. In BD, Trips are not associated with locations, and that is ok. You can query your sightings by trip or by any other criteria you wish. You can even combine criteria to zero in on what you are looking for. 

    For your possible solutions, I am not sure that I see the problem. I am not aware of any specific problem in the import and mapping for BB sightings to BD sightings. Perhaps if you gave me one specific sighting example and we could work through it. Your solutions #3 and #4 above are certainly workable. But I don't see the need to do the edit/change on any of the imported records. So with the one sighting example, set the stage with what the data was in BB, how it imported into BD, and what it is that you are wanting to do with that sighting that is causing the issue.

    Let's start here please,

    Thanks,

    Jeff

    Birder's Diary Technical Support
    tapaculo
    Yellow Warbler
    Yellow Warbler
    Posts:19


    --
    02/15/2019 5:55 PM

    Thanks for the quick response. I agree that BD did what it should have done with respect to import and mapping, and all it could have, given how BB organized the data. This was never a complaint about Birder's Diary.

    Since I wasn't clear, here is an example of what my problem is: if I export my records to eBird, all my thousands of sightings in Ecuador, for example -- with the precise location for each of them indicated in BB via the trip description -- will be exported with the location simply "Ecuador". Right? If that's not true, let me know before I go any further.

    By the way, the one example of a "mis-conversion" of Location in my own records was BB's GS for the Galapagos... they got imported as South Georgia Island. That was fixed easily with your "mass edit" capability.

    tapaculo
    Yellow Warbler
    Yellow Warbler
    Posts:19


    --
    02/15/2019 8:37 PM
    That's a nice feature, to automatically add all the "subunits" of the initial locations. So more detailed locations that a user creates should be "children" of one of those.

    The next step would be to change the location for sightings to one of the new "detailed locations". I will do that one trip at a time (since each trip and all its sightings will relate to just one of the new detailed locations), doing a mass edit. That will be add some efficiency.

    However, I have over 500 trips, so it would still be quite laborious. Something that could make this more efficient would be (please bear with me!) to export all the data, probably to a CSV file, modify the location field in each record to one of the new detailed locations, and then import that modified data to replace the records in Birder's Diary. Does something like that seem feasible? (I can explain what I would do "off-line" to modify the locations, but that's irrelevant to whether it makes any sense to "re-create" the data in BD via an import.)

    Thanks again for your patience and help. I still think many or most ex-BirdBase users would have to do something like this...

     

    p.s. These are displaying in the wrong order. I submitted this at ~9:40 CST on 2/15 as a response to an entry of yours (with the BD screen capture) with a timestamp of 11:41.

     

    Jeff Jones
    Great Gray Owl
    Great Gray Owl
    Posts:8297


    --
    02/15/2019 9:26 PM

    Hi Steve,

    Whatever the location is in BD, is what will be exported to eBird. I assume that is 'Ecuador' in this example. You can export your sightings to an eBird CSV file and then open that file to examine what got exported. This is all before you start the import into eBird on the eBird.org web site. 

    Let me know,

    Jeff

    Birder's Diary Technical Support
    tapaculo
    Yellow Warbler
    Yellow Warbler
    Posts:19


    --
    02/15/2019 10:56 PM
    OK, so we are on the same page, thanks. What I want to do is get a more detailed location into the Location field. e.g. Mindo instead of just Ecuador or "Sax-Zim bog" instead of just Minnesota. (I assume many other BirdBase refugees would have the same need).

    So I believe it would be necessary to do the following.

    > [1] Under Location Maintenance, create those locations, e.g. Mindo.

    > [2] For each new location, designate the country (in this case, Ecuador) and maybe other layers of Parent and Child locations. (These first two steps are what I have done for new sightings entered within Birder's Diary.)

    > Replace the current Location, e.g. Ecuador or Minnesota, with one of the more specific locations that were created. One more efficient way to do that, instead of doing one sighting at time, would be to [3] display all the sightings for each Trip within View/Edit Sightings -- because those are all from the same "detailed location" -- and [4] do a mass edit.


    If that's all correct, I do have one more idea about how to do that more efficiently that one Trip at a time, and would appreciate opinions on that...
    Jeff Jones
    Great Gray Owl
    Great Gray Owl
    Posts:8297


    --
    02/15/2019 11:41 PM

    Hi Steve,

    1. Yes - add the needed locations under each state/country. NOTE: Counties and sub-locations are already defined for states and countries.

      So you would add the counties of MN before adding Sax-Zim. And for Ecuador you would add the main provinces.
    2. And then mass edit sightings by location. 

     

    Let me know,

    Jeff

    Birder's Diary Technical Support
    Jeff Jones
    Great Gray Owl
    Great Gray Owl
    Posts:8297


    --
    02/16/2019 12:13 PM
    Posted By tapaculo on 02/15/2019 8:37 PM
    That's a nice feature, to automatically add all the "subunits" of the initial locations. So more detailed locations that a user creates should be "children" of one of those.

    The next step would be to change the location for sightings to one of the new "detailed locations". I will do that one trip at a time (since each trip and all its sightings will relate to just one of the new detailed locations), doing a mass edit. That will be add some efficiency.

    However, I have over 500 trips, so it would still be quite laborious. Something that could make this more efficient would be (please bear with me!) to export all the data, probably to a CSV file, modify the location field in each record to one of the new detailed locations, and then import that modified data to replace the records in Birder's Diary. Does something like that seem feasible? (I can explain what I would do "off-line" to modify the locations, but that's irrelevant to whether it makes any sense to "re-create" the data in BD via an import.)

    Thanks again for your patience and help. I still think many or most ex-BirdBase users would have to do something like this...

     

    p.s. These are displaying in the wrong order. I submitted this at ~9:40 CST on 2/15 as a response to an entry of yours (with the BD screen capture) with a timestamp of 11:41.

     


    Wow - that is weird. Let's see where this shows up.

    Don't many of your BB Trips have the same location in BD? If so, then you just pull up all sightings by Location and mass edit them. Or am I missing something. Do you actually have over 500 different locations that you have sightings for? My understanding is that many of your BB Trips are for the same location.

    But, to answer your question, I really can see any way that exporting your sightings (via File | Import/Export..., Sightings Tab) and then editing the exported file and re-importing those sightings would be faster than what I suggest above. Also, it would create duplicate sightings that you would need to delete. 

    fyi

    Jeff

    Birder's Diary Technical Support
    tapaculo
    Yellow Warbler
    Yellow Warbler
    Posts:19


    --
    02/16/2019 2:27 PM
    An ex-BB user cannot "pull up all sightings by Location and mass edit them", because sightings imported from BB -- mine and, I believe, those of every other BB convert -- have a Location that is either a country or a US state. (Well, a few exceptions where BB didn't use countries, like Sulawesi or Galapagos.) That's the problem.

    So I have to use Trips, which were also imported and to which each sighting is linked. You're right, I haven't been to 500 sites/places, there are multiple trips to the same place, but I didn't write only a location name in the BirdBase descriptions for each trip. E.g. here are some descriptions for several of the trips to the same location:
    High; Papallacta pass. 12/26, 29
    Papallacta, PI; High
    Guango to Papallacta, Temp-High, ~3000-4000m, 1/24-25

    (By the way, though I've checked the box to "receive email notifications for this topic", and it shows at the top that I am subscribed, I haven't gotten any emails when you have replied. )
    Jeff Jones
    Great Gray Owl
    Great Gray Owl
    Posts:8297


    --
    02/16/2019 2:54 PM

    Are those comments in your example or actual Trip names? If they are comments, then you can search on them like this...

    If trips, then you cannot. Did you keep comments with anything that could tag a sighting as to place?

    Concerning notifications: I am getting email notifications so I don't think there is a problem on this end. Did you check your spam/junk folders? That is the first place to start. Let me know please.

    Jeff

    Birder's Diary Technical Support
    Jeff Jones
    Great Gray Owl
    Great Gray Owl
    Posts:8297


    --
    02/16/2019 3:00 PM
    Note: In v5.1, development in progress now, I have updated the BB import to include the Trip Name and Date in the comments field.

    fyi

    Jeff
    Birder's Diary Technical Support
    tapaculo
    Yellow Warbler
    Yellow Warbler
    Posts:19


    --
    02/16/2019 10:08 PM
    You wrote "Are those comments in your example or actual Trip names? If they are comments, then you can search on them..."

    No, unfortunately the comments come from the "sighting notes" in individual BB sightings, and my examples are trip profile descriptions. (I can't figure out how to paste a screen capture here, so will send you an email.) Perhaps some BB users did put the specific location in each sighting, but it was much more efficient to put it in the description for the trip, so that each sighting could inherit it from the trip just like the it inherits the date and country/state from its trip. To display sightings, BB allows the user to specify required text in the sighting's trip description/profile as well as its date.

    By the way, if you are doing updates to the import, it might be helpful to make each imported Trip a child of its associated country or state. Given how you already deal with those fields, that would be easy for you.

    Since as you point out, manipulating the data off-line and importing that would create duplicate records (unless I deleted all the existing records first!), I'll just have to update the locations within Birders Diary via the four steps I noted above (I went back and added numbers) for each trip.

    About notifications, they are not in my Outlook "Junk e-mail" folder, nor in the Google Mail in the cloud.

    Steve  2/16/2019 11:06:06 PM CST

    Jeff Jones
    Great Gray Owl
    Great Gray Owl
    Posts:8297


    --
    02/17/2019 7:45 AM

    Hi Steve,

    I double-checked the email for this profile and it matches the email address that you sent me pics with. So that is good. I am not getting any bounced email notifications, so your server is not rejecting the emails. As I mentioned earlier, I and other users are getting email notifications so I believe that the notification subsystem is doing what it should be doing. This only leaves between your server and your email.

    I responded to an email that you sent to me. Please let me know if you got that.

    Thanks,

    Jeff

    Birder's Diary Technical Support
    You are not authorized to post a reply.


    Support Tools

     TitleOwnerCategoryModified DateSize 
    Screen Capture ToolJeff Jones 3/8/2016UnknownDownload
    GreenShot - Advanced Screen Capture ToolJeff Jones 12/18/2013UnknownDownload