The database I am building to replace our Dissemination Time database is well underway. If you access it now it is back to 2021. However, I accidentally discovered an interesting problem. The OldLoads file from 4/15/2022 is empty. Here is a link to the archive page if you want to visit it. Below is a screenshot of the actual link on the page:

Not a huge file, but not insignificant. I did not know it was empty because I did not anticipate that one of these files would be empty when we downloaded it. I anticipated that the connection would break or the file would download in a manner that corrupted it but I did not anticipate that it would have no content. I only discovered this problem because someone was checking some results and asked me why there were not any results for an 8-K filing they expected as they were spot-checking their results.

Now here is the interesting piece, I think we successfully downloaded that archive in the summer of 2022 but we were not able to access the time field then. I think that because we have the filings listed in the original dissemination time database but we don’t have a value for the DISSEMINATION_TIME and for many of the filings we don’t have a value for FORM.

This is interesting, and if I made a note of that during processing, then that note was lost to time.

I addressed the missing 8-K filings for our client by using the index files to identify filings made on 4/15/2022 and then pulled the filings and sent them a file with the ACCEPTANCE-DATETIME from the header. I can’t see any other way to resolve this.

I think we are just going to have to populate the database for 4/15/2022 filings with what we can glean from the header associated with the filings. We will add a code field to the database to indicate that the data for these rows came from a different source. Of course the problem is that any deletions or other non-standard activity for that day will not be captured.

I hate to admit that this is the second time that date has flummoxed us. The simple truth is that when we set out to do something we try to imagine every error that can occur in the process. Evidently our imagination isn’t good enough. The bad thing is an empty file is an issue that we have experienced in the past, though in a different context. I can’t grab an example, but there are EDGAR accession.txt files that are empty. We discovered that possibility because our parsing engine expects to find document blocks demarcated by SGML tags <DOCUMENT> . . .</DOCUMENT> and so the first time we encountered an empty one, the parser crashed with a Not Found error. We learned from that experience that it was necessary to anticipate that problem and code for it. But the code that processes these files to extract the date did not throw an error because it iterates through the file, and the fact that there was nothing to iterate through was not a problem.

I have been accused in the past of not having an imagination. I guess this is evidence of the truth of that.