From Financial Statement Notes to Research Data

We think this is one of the most consequential improvements we have made to the directEDGAR platform.

For years, researchers have used directEDGAR to search specific SEC document types and filter by CIK at scale. Finding the right disclosure is an essential first step. Collecting the amounts, dates, and other details for a research dataset can still require considerable work. Our new feature searches financial statement notes and exports their text and tagged facts, taking on much of that next step. If you’re ready to put it to work, the Knowledge Base guide covers how to search, export results, and use the data.

The idea grew out of a client’s request for structured data from financial statement footnotes. It also brought me back to my dissertation, when I searched microfiche copies of 10-Ks for acquisition disclosures and recorded purchase price allocations on my first laptop. I know how much time can go into finding and transcribing those observations.

The new tool searches notes curated from iXBRL 10-K filings and exports the matching disclosures as HTML snippets. It also collects their tagged facts into separate CSV files for tables and narrative text. Each fact retains its tag, value, unit, available period and dimension information, and a reference back to its source snippet. Narrative facts also retain the sentences identified as containing them.

Finding the useful facts within a matching note

For a test, I searched for purchase price AND allocation AND goodwill. One match was Iron Mountain’s summary of significant accounting policies, which spans almost 15 pages in its 10-K filed in 2021. The exported snippet supplied 504 tagged fact rows from tables and another 80 from text. The disclosure contained my search terms and some useful facts, but its broad coverage also brought in many facts outside my data collection goal.

That is where the individual fact tags help. I can distinguish goodwill acquired in an acquisition from goodwill impairment, and filter out facts about leases, depreciation, or other subjects. The search brings me to the disclosure; the tags help me identify useful observations within it. The source HTML remains available so I can check what the amounts mean.

One sentence, five tagged facts

The same search found the business combinations note in Quest Diagnostics’ 2020 Form 10-K, filed in 2021. It describes Quest’s acquisition of the remaining 56% interest in Mid America Clinical Laboratories on August 1, 2020. The note reports the preliminary allocation in this sentence:

Based on the preliminary purchase price allocation, which may be revised as additional information becomes available during the measurement period, the assets acquired and liabilities assumed consist of $84 million of goodwill (of which $47 million is tax-deductible), $74 million of intangible assets, $11 million of working capital and $11 million of property, plant and equipment.

Each of the five amounts was individually tagged, so the text facts file provides five rows, with the complete sentence alongside each one. The underlying XBRL value for goodwill is reported in dollars as 84000000, while the displayed 84 is preserved separately.

The fact’s XBRL context also supplies the acquisition date and identifies the acquisition. Those details are available alongside the amount, even though the allocation sentence itself does not state the date.

This is your research. We preserve the original sentence, displayed values, and source HTML so you can check the amounts against the disclosure and understand their meaning as you integrate the data into your models.

For example, the tag on the $47 million row identifies the amount as goodwill expected to be tax-deductible:

us-gaap:BusinessAcquisitionPurchasePriceAllocationGoodwillExpectedTaxDeductibleAmount

The sentence confirms that this amount is included in the $84 million of total goodwill, rather than another allocation amount to add to it.

Try it with your research question

We expect this to shorten the path from finding a disclosure to assembling a research dataset. You will still need to select the relevant facts and review their context, but much of the collection work is already done. The initial databases cover 2021 and 2022, and we are adding coverage incrementally.

For access instructions, search examples, continuation steps, and descriptions of the CSV fields, see Search Financial Statement Notes and Export Tagged Facts in the Knowledge Base. Please share your experiences and suggestions—we expect the tool to evolve as you put it to work.

One company connected to three CIKs, illustrating the risks of missing filings and duplicate filings.

A Database Access Problem Raised a Larger CIK Question

One company connected to three CIKs, illustrating the risks of missing filings and duplicate filings.

I received an email from a PhD student who was unable to open the 8Kmeta.db database.

The cause turned out to be a mistake on my end. While working on the database, I had inadvertently changed its SQLite journal mode to Write-Ahead Logging, or WAL mode. That setting was incompatible with the way we distribute the database for read-only use, so our application could not open it.

The source of the problem was not immediately obvious to me. To keep the student moving forward while I investigated, I asked her to send me the list of CIKs in her sample. I extracted the relevant 8-K metadata and sent her the results.

Once I identified the problem, I returned the database to its normal journal mode. While doing so, another question occurred to me: had any of the companies in her sample experienced a corporate event that resulted in a change of CIK?

I used our CIK_MAPPING database to check. Thirteen CIKs in her sample were associated with another CIK:

BASE_CIKBase entityRELATED_CIK_1Related entity
48039HollyFrontier Corp.1915657HF Sinclair Corp.
701221Cigna Holding Co. (formerly Cigna Corp.)1739940The Cigna Group
808362Baker Hughes Holdings LLC (formerly Baker Hughes Inc.)1701605Baker Hughes Co.
863821Restoration Hardware Inc.1528849RH
904080Talos Petroleum LLC (formerly Stone Energy Corp.)1724965Talos Energy Inc.
1279176CPI International, Inc.1515003CPI International Holding Corp.
1396009Vulcan Materials Co.103973Legacy Vulcan Corp. (formerly Vulcan Materials Co.)
1465112DIRECTV944868DIRECTV Group Inc.
1532063Express Scripts Holding Co.885721Express Scripts Inc.
1613665Great Western Bancorp, Inc.1088381Great Western Bancorporation Inc.
1627014Black Knight, Inc.1704177Black Knight, Inc.
1645494ARRIS International plc1141107ARRIS Group Inc.
1671013Cardtronics plc1277856Cardtronics plc (formerly Cardtronics Inc.)

The BASE_CIK and RELATED_CIK_1 labels reflect their positions in our mapping database. They should not be interpreted as consistently identifying the older and newer entities. In some rows, the related CIK is the successor. In others, it is the predecessor.

I used the values in RELATED_CIK_1 to search for additional 8-K filings. That search identified approximately 1,800 observations that were not returned when I searched only the original CIKs.

The original search produced 227,648 observations, so the additional records increased the result by less than 1%. In aggregate, that is not a dramatic difference.

However, the missing observations may not be random. A CIK change is often associated with a merger, reorganization, change in corporate structure, or another potentially meaningful event. Even a relatively small number of omitted filings could matter depending on the research question, sample construction, and timing of the events being studied.

The fact that I did not immediately think to check for related CIKs has made me reconsider how we should handle these relationships in the database.

One possibility is to duplicate rows associated with a related CIK. For example, for every observation filed under CIK 1915657, we could add a second row in which the searchable CIK is 48039. A researcher searching only for 48039 would then retrieve observations associated with both identifiers without first having to consult the mapping database.

We could add fields identifying the CIK under which the filing was actually submitted and indicating that an additional row was created because of a CIK relationship. This would preserve the provenance of the observation even though the row had been duplicated for search purposes.

However, that approach introduces a different problem. If a researcher includes both CIKs in a sample, the same observation will be returned twice. The researcher would then have to identify and remove the duplicate after retrieving the data.

Disney illustrates how quickly this can become more complicated than a simple two-CIK relationship. The public parent company has used three CIKs across the SEC’s electronic filing period:

CIKEntity during the relevant periodFirst 8-KLast 8-K
29082The Walt Disney Company; now Disney Enterprises Inc.1994-03-141996-04-05
1001039The Walt Disney Company; now TWDC Enterprises 18 Corp.1995-12-012019-03-20
1744489The Walt Disney Company, current parent2019-03-20Present

The overlapping dates are not an error. Filings associated with a corporate transition can occur under more than one CIK during the transition period.

For purposes of parent-company continuity, the cleaner Disney sequence is:

  • CIK 29082 through February 9, 1996;
  • CIK 1001039 from February 9, 1996 through March 20, 2019; and
  • CIK 1744489 from March 20, 2019 to the present.

The SEC filing history shows CIKs 29082 and 1001039 appearing together during the first transition. Disney’s March 20, 2019 reorganization filing explains the later transition: CIK 1001039, formerly The Walt Disney Company, became TWDC Enterprises 18 Corp., while the holding company using CIK 1744489 became the current Walt Disney Company.

If we duplicate Disney observations so that a search for any of the three CIKs returns the entire time series, a researcher who includes all three CIKs could receive some observations three times. Additional fields could make those duplicate or triplicate records identifiable, but the researcher would still need to understand why they exist and remove them when appropriate.

This leaves us with an uncomfortable choice.

If we do not duplicate the rows, a researcher who searches only the original sample CIKs may unknowingly omit filings made under related CIKs. Avoiding that problem requires the researcher to recognize the possibility of CIK changes, consult the mapping database, and conduct additional searches.

If we do duplicate the rows, searches become more complete by default, but researchers may unknowingly count the same observation more than once. We can add fields and documentation to help them identify the duplicates, but we cannot force the existing retrieval tool to remove them automatically.

The two approaches therefore place different burdens on the researcher. One creates a risk of missing observations; the other creates a risk of double counting them.

For this student’s sample, searching the related CIKs produced approximately 1,800 additional observations beyond the 227,648 returned using the original CIKs. That may or may not affect her conclusions. Nevertheless, the incident exposed a gap in the current workflow and a larger question about how directEDGAR should support longitudinal research when the identifiers used by public companies change.

I do not think there is an easy solution. At present, I am trying to decide whether it is better for the database to favor completeness—with clearly identified duplicate rows—or to preserve one row per observation and require researchers to account for CIK changes before retrieving their data.

If you have an opinion, I would love for you to share it.