I received an email from a PhD student who was unable to open the 8Kmeta.db database.
The cause turned out to be a mistake on my end. While working on the database, I had inadvertently changed its SQLite journal mode to Write-Ahead Logging, or WAL mode. That setting was incompatible with the way we distribute the database for read-only use, so our application could not open it.
The source of the problem was not immediately obvious to me. To keep the student moving forward while I investigated, I asked her to send me the list of CIKs in her sample. I extracted the relevant 8-K metadata and sent her the results.
Once I identified the problem, I returned the database to its normal journal mode. While doing so, another question occurred to me: had any of the companies in her sample experienced a corporate event that resulted in a change of CIK?
I used our CIK_MAPPING database to check. Thirteen CIKs in her sample were associated with another CIK:
| BASE_CIK | Base entity | RELATED_CIK_1 | Related entity |
|---|---|---|---|
| 48039 | HollyFrontier Corp. | 1915657 | HF Sinclair Corp. |
| 701221 | Cigna Holding Co. (formerly Cigna Corp.) | 1739940 | The Cigna Group |
| 808362 | Baker Hughes Holdings LLC (formerly Baker Hughes Inc.) | 1701605 | Baker Hughes Co. |
| 863821 | Restoration Hardware Inc. | 1528849 | RH |
| 904080 | Talos Petroleum LLC (formerly Stone Energy Corp.) | 1724965 | Talos Energy Inc. |
| 1279176 | CPI International, Inc. | 1515003 | CPI International Holding Corp. |
| 1396009 | Vulcan Materials Co. | 103973 | Legacy Vulcan Corp. (formerly Vulcan Materials Co.) |
| 1465112 | DIRECTV | 944868 | DIRECTV Group Inc. |
| 1532063 | Express Scripts Holding Co. | 885721 | Express Scripts Inc. |
| 1613665 | Great Western Bancorp, Inc. | 1088381 | Great Western Bancorporation Inc. |
| 1627014 | Black Knight, Inc. | 1704177 | Black Knight, Inc. |
| 1645494 | ARRIS International plc | 1141107 | ARRIS Group Inc. |
| 1671013 | Cardtronics plc | 1277856 | Cardtronics plc (formerly Cardtronics Inc.) |
The BASE_CIK and RELATED_CIK_1 labels reflect their positions in our mapping database. They should not be interpreted as consistently identifying the older and newer entities. In some rows, the related CIK is the successor. In others, it is the predecessor.
I used the values in RELATED_CIK_1 to search for additional 8-K filings. That search identified approximately 1,800 observations that were not returned when I searched only the original CIKs.
The original search produced 227,648 observations, so the additional records increased the result by less than 1%. In aggregate, that is not a dramatic difference.
However, the missing observations may not be random. A CIK change is often associated with a merger, reorganization, change in corporate structure, or another potentially meaningful event. Even a relatively small number of omitted filings could matter depending on the research question, sample construction, and timing of the events being studied.
The fact that I did not immediately think to check for related CIKs has made me reconsider how we should handle these relationships in the database.
One possibility is to duplicate rows associated with a related CIK. For example, for every observation filed under CIK 1915657, we could add a second row in which the searchable CIK is 48039. A researcher searching only for 48039 would then retrieve observations associated with both identifiers without first having to consult the mapping database.
We could add fields identifying the CIK under which the filing was actually submitted and indicating that an additional row was created because of a CIK relationship. This would preserve the provenance of the observation even though the row had been duplicated for search purposes.
However, that approach introduces a different problem. If a researcher includes both CIKs in a sample, the same observation will be returned twice. The researcher would then have to identify and remove the duplicate after retrieving the data.
Disney illustrates how quickly this can become more complicated than a simple two-CIK relationship. The public parent company has used three CIKs across the SEC’s electronic filing period:
| CIK | Entity during the relevant period | First 8-K | Last 8-K |
|---|---|---|---|
| 29082 | The Walt Disney Company; now Disney Enterprises Inc. | 1994-03-14 | 1996-04-05 |
| 1001039 | The Walt Disney Company; now TWDC Enterprises 18 Corp. | 1995-12-01 | 2019-03-20 |
| 1744489 | The Walt Disney Company, current parent | 2019-03-20 | Present |
The overlapping dates are not an error. Filings associated with a corporate transition can occur under more than one CIK during the transition period.
For purposes of parent-company continuity, the cleaner Disney sequence is:
- CIK 29082 through February 9, 1996;
- CIK 1001039 from February 9, 1996 through March 20, 2019; and
- CIK 1744489 from March 20, 2019 to the present.
The SEC filing history shows CIKs 29082 and 1001039 appearing together during the first transition. Disney’s March 20, 2019 reorganization filing explains the later transition: CIK 1001039, formerly The Walt Disney Company, became TWDC Enterprises 18 Corp., while the holding company using CIK 1744489 became the current Walt Disney Company.
If we duplicate Disney observations so that a search for any of the three CIKs returns the entire time series, a researcher who includes all three CIKs could receive some observations three times. Additional fields could make those duplicate or triplicate records identifiable, but the researcher would still need to understand why they exist and remove them when appropriate.
This leaves us with an uncomfortable choice.
If we do not duplicate the rows, a researcher who searches only the original sample CIKs may unknowingly omit filings made under related CIKs. Avoiding that problem requires the researcher to recognize the possibility of CIK changes, consult the mapping database, and conduct additional searches.
If we do duplicate the rows, searches become more complete by default, but researchers may unknowingly count the same observation more than once. We can add fields and documentation to help them identify the duplicates, but we cannot force the existing retrieval tool to remove them automatically.
The two approaches therefore place different burdens on the researcher. One creates a risk of missing observations; the other creates a risk of double counting them.
For this student’s sample, searching the related CIKs produced approximately 1,800 additional observations beyond the 227,648 returned using the original CIKs. That may or may not affect her conclusions. Nevertheless, the incident exposed a gap in the current workflow and a larger question about how directEDGAR should support longitudinal research when the identifiers used by public companies change.
I do not think there is an easy solution. At present, I am trying to decide whether it is better for the database to favor completeness—with clearly identified duplicate rows—or to preserve one row per observation and require researchers to account for CIK changes before retrieving their data.
If you have an opinion, I would love for you to share it.