Sunday, November 19, 2017

The World's Premier Listing Software
for Birders and Naturalists!

Celebrating 22 Years Serving Birders
and Naturalists around the World!

Search this Site and the Forum Contents

Birder's Diary Forums

importing data with missing or bad date field
Last Post 02/02/2017 8:59 AM by Jeff Jones. 4 Replies.
  • Status:
  • Test\u0000:
Printer Friendly
  •  
  •  
  •  
  •  
  •  
Sort:
PrevPrev NextNext
You are not authorized to post a reply.
Author Messages
Bert Frenz
Great Gray Owl
Great Gray Owl
Posts:214


--
01/31/2017 10:35 AM

    I imported some data from the Smithsonian Collection.  I downloaded a CSV from the collection and then arranged columns to match BD.  However, I failed to notice some errors in the date field.  Here is how BD processes this data:

    No date – is processed as current date

    1888 – is processed as 3/2/1905

    Sep 1880 – is processed as 9/1/1880 (which is probably okay, but I would have chosen 9/15/1880)

    (Dec 18--) – the entire record is skipped

    While obviously I should be more careful about what I am entering (garbage in – garbage out) I think BD could be a bit more helpful.  First of all, none of these data errors were mentioned in the report file.  It seems at the very least BD should put a message in the report file if it skips a record.  More helpful would be that it prompts the user if the data is incomplete.

    If you would like to test this yourself, I have attached a CSV file that includes some of the records with bad date fields.

    Bert

    Attachments
    Jeff Jones
    Great Gray Owl
    Great Gray Owl
    Posts:7807


    --
    01/31/2017 11:17 AM

    Hi Bert,

    Thanks for taking the time to give examples and post the bad import file.

    I will take a look at this and attempt to improve on error handling.

    I will update you here with what I find.

    Jeff

    Birder's Diary Technical Support
    Jeff Jones
    Great Gray Owl
    Great Gray Owl
    Posts:7807


    --
    01/31/2017 12:35 PM

    Hi Bert,

    I use Microsoft's library for interpreting dates. 

    I have no idea why it thinks the string of "1888" is "3/2/1905", but it does.

    But for the rest, it seems to take reasonable conversions.

    I have updated BD (for v5.0 due out soon) to log all errors with dates and other things to the log file, and notify you properly at the end that there were errors and to review the log file. The log file will contain details on the records that were unable to be processed.

    Take care,

    Jeff

    Birder's Diary Technical Support
    Bernard Reddy
    Yellow Warbler
    Yellow Warbler
    Posts:10


    --
    02/01/2017 8:03 PM
    No doubt "3/2/1905" is 1888 days after day 0 for Access.
    Jeff Jones
    Great Gray Owl
    Great Gray Owl
    Posts:7807


    --
    02/02/2017 8:59 AM

    Ah yes Bernard! That is exactly what that is.

    Day 0 is "12/30/1899"!

    Always an answer for these sorts of things. Just have to dig to figure it out.

    Jeff

    Birder's Diary Technical Support
    You are not authorized to post a reply.


    Support Tools

     TitleOwnerCategoryModified DateSize 
    Screen Capture ToolJeff Jones 3/8/2016UnknownDownload
    GreenShot - Advanced Screen Capture ToolJeff Jones 12/18/2013UnknownDownload