There are a number of papers in the accounting literature that have described using directEDGAR to collect inventory write down data beginning with the Allen, Larson and Sloan 2012 Journal of Accounting and Economics paper. Historically, the collection for this data has been based on running key word searches in 10-K filings to identify potentially relevant content and then reviewing the extracted context to determine if it should be included in the sample.

Another paper that used very similar data was Ming, Markov and Shu’s 2023 Accounting Review, Motivational Optimism and Short-Term Investment Efficiency, I constructed a search that I think was close to theirs (DOCTYPE contains(10k)) and inventory w/5 (write* or obsolescence or obsolete or charge* or revaluation or provision*) pre/15 === and applied it to the 2025 10-K archive (the === signs force the search to only return results that are followed by at least three digits.). The results seemed consistent with what they reported (even though their data collection covered an earlier time period). Here is a screenshot of the search results.

Based on reading their paper and how I would collect the same data I would then run a SummaryExtraction to get the context into a CSV file to then scan and code the results.

I then wondered, could we accelerate this using the XBRL databases we have created? One challenge with using the databases is that I would either have to identify the relevant row labels or try to identify the way the data should be tagged. I decided to try the tag route first. Because I could not possibly expect filers to use a constrained set of tags I decided to make an assumption that the actual XBRL tags had to include the word inventory and then at least one of the words in my (their) original search. Here is the search I ran with the Query tool ((name LIKE ‘%charge%’) OR (name LIKE ‘%provision%’) OR (name LIKE ‘%obsol%’) or (name LIKE ‘%write%’) or (name LIKE ‘%reval%’) ) AND (name LIKE ‘%inventory%’). To be clear, I am requiring the tag used for the data to have the word inventory and at least one of the other character sequences.

Here is a screenshot of the results

I saved the results to review them and compare to the full-text search results. There were 2,298 total observations with 212 unique tags. Not all of the results were relevant. Here is a sample of some of the tags.

aaon:InventoryValuationReservesWriteOffs
iex:CostOfGoodsAndServicesSoldExcludingInventoryStepUpCharges
iex:FairValueInventoryStepUpCharges
cgnx:ExcessAndObsoleteInventoryCharges
icui:WarrantyAndReturnReserveInventoryChargedToOtherAccounts
icui:WarrantyAndReturnReserveInventoryWriteOffs
ftk:ProvisionForExcessAndObsoleteInventory
gti:CostOfGoodsAndServicesSoldExcludingInventoryWriteDown
nvax:ProvisionForExcessAndObsoleteInventory
lctc:ProvisionForInventoryObsolescence
duot:InventoryWriteoff
ngs:InventoryAllowanceAllowanceForObsolescence
ngs:InventoryWriteOffs
ew:DisposalGroupIncludingDiscontinuedOperationInventoryWriteDown
cvgw:ProvisionForAdvancesOnInventoryPurchases
arw:InventoryReversalOfWriteDown
clne:ProvisionForDoubtfulAccountsNotesAndInventory
acls:ProvisionForExcessAndObsoleteInventory

I see a really clean work-flow following from this. I think the first thing I would do is separate the tags, identify the irrelevant ones (cvgw:ProvisionForAdvancesOnInventoryPurchases) and filter those out. I won’t go further because I imagine many of you can already imagine the next steps. The point is that accessing this data significantly accelerated when the data is tagged.

The key is to be creative with how you initially filter for relevant tags. If I had to consider every tag with the word INVENTORY (there were 21,985 tags with the word inventory) the review would be overwhelming. But by requiring the word INVENTORY and then at least one other of the word roots that describe the possibility of a write-down I am evaluating a much smaller set of results for relevance.

Of course, the results are not likely to be exhaustive. I found some interesting omissions. Here is a screenshot from Blink Charging’s full-text search results:

This data was not in the results. I searched for it directly using the Query Tool and discovered that the tag was ProvisionForOtherLosses. But I noticed that they did use relevant text in the original row label as you can see in the screenshot above. I think the next step is to get creative in a similar manner using the orig_row_label field which is also fully searchable. I ran this search (orig_row_label LIKE ‘%inven%’) AND ( (orig_row_label LIKE ‘%obsol%’) OR (orig_row_label LIKE ‘%slow%’) or (orig_row_label LIKE ‘%write%’) ).

While there is sure to be duplication in the second run, that is easy to handle because you can filter out all of the tags that were in the first run.

I am not going to assert that this is going to provide complete results. But I believe that if a third round of filtering is needed it should be minimal. Finally, if there is still data to be collected, we can fallback to the full-text search after filtering out the CIKs from the filers whose tagged data was useful. Since only the financial statements and notes are tagged, if the filer made the disclosure only in the MDA without separately reporting the amount in the financials or notes then a full-text search would be the only way to identify the value.

I want to wrap this up by adding a bit of marketing. The real benefit of our platform comes from the visibility we give you into the filings. With that initial search I could easily move through the collection of filings to get a sense of my results. I instantly know if I need to expand (or contract) the search. And then within the same interface I can immediately access the tagged data. The ability to filter tags without requiring their adherence to the taxonomy provided significant benefits. Again, once I saved the results and then looked back at my search results I was ready to filter on the row labels.

When Paul Atkins, Chair of the U.S. Securities and Exchange Commission, spoke at the Texas A&M University School of Law Corporate Law Symposium, he discussed the growing complexity of SEC disclosure rules.

Referring to executive compensation disclosure under Item 402 of Regulation S-K, he said:

“The rule today has morphed into a Frankenstein monster beyond recognition… disclosure intended to inform can instead overwhelm.”

Today, Berkshire Hathaway filed its DEF 14A proxy statement — and it provides a fascinating example of that concern. Because of Item 402(u), the company must disclose its CEO pay ratio.

Yet Warren Buffett has earned $100,000 annually for more than 40 years, with no bonus and no equity compensation.

Nevertheless, Berkshire still had to estimate the median compensation of 387,815 employees across more than 60 operating groups in order to compute the required ratio.

The result:

• CEO compensation: $389,488 ($289,488 of this is security)
• Median employee compensation: $93,709
• Pay ratio: 4.16 to 1

In the proxy statement, Berkshire explicitly questioned whether the cost/benefit of complying precisely with Item 402(u) provides meaningful information to shareholders. While the cost is probably inconsequential to the company, I suspect the attention required for this is non-trivial since they run such a tight ship.

If the SEC is serious about rationalizing, simplifying, and modernizing Regulation S-K, this raises a straightforward question:

Should CEO pay ratio disclosure be required when total CEO compensation is below $1 million?

For companies like Berkshire, the disclosure may add complexity without materially improving investor understanding.

And Berkshire is not unique — more broadly, there are a non-trivial number of public companies where CEO compensation is below $1 million, suggesting this issue may not be unique.

For researchers interested in examining these disclosures, Mr. Buffett’s compensation — along with the all of the other proxy filers who made their filing today — will be available around midnight tonight for analysis in the directEDGAR Executive Compensation database.

We completed the out-of-cycle update to the Tax-Recon database. Manish did a heroic job.

First off, the distribution so far – 988 firms that have filed their 10-K as of yesterday (3/2) have adopted the requirements of ASU-2023-009 on a prospective basis. 760 adopted on a retrospective basis. When you access the database – remember that ASU-2023-009 was required for FY beginning after 12/15/202. If you filter on 2026 filings you will find no ASU_STANDARD flag for 214 accession numbers – so far one of these is a REIT, the other 213 had fiscal years ending in October, November and even early December so they are just now subject to the new disclosure requirements.

While we are investigating ways to remove some of the flags – their existence should not keep you from moving forward. For example, there are 245 cases where we have inserted a CHECK_TOTAL flag. It looks like the vast majority of these are because the registrant reports a subtotal – that is not labeled as such. Here is an example from GATX’s 10-K –

The Foreign tax effects line represents the net foreign tax effect on the dollars and rate. When we test the addition (and we do so to confirm that we are not missing something that is hidden) we would double count those values as we would include all of the values above the subtotal. We are not changing anything – the reported totals are available in the database with the values you see. But because we don’t know we have flagged this table as one that requires review.

I have to be honest – I am not sure yet how we should handle these. One option to to add an additional flag to indicate that row is a subtotal. Another option is to delete the row.

The MULTI-TABLES flag is harmless – it is the starting place for our analytical analysis of the ASU-2023-009 adoption method. We do much more than test for the existence of multiple tables but it is part of that routine. We left that flag in the client facing database because it can be helpful if you report a concern.

With respect to the ASU_2023_09 flags. The flag ASU_2023_PROSPECTIVE identifies the data in the filing that conforms to the new disclosure standard for the CURRENT year. The flag ASU_2023_PROSPECTIVE_HISTORICAL identifies the historical (generally prior two years) data that will generally be presented in only one unit. But again, be careful, there are filers that presented data in both percentage and monetary units that adopted the standard prospectively, Boeing is a good example). Boeing has two tables, one that conforms to the new standard (and has both dollars and percentages) and their historical disclosure that also reports both dollars and percentages. The flag ASU_2023_RETROSPECTIVE identifies the filings where the registrant adopted the ASU on a retrospective basis.

I ran some very rough analysis – there are about 30% more labeled data rows when firms meet the ASU 2023-009 requirements as compared to their historical disclosures. There are lots of other interesting features in the data but I know some of our clients are focused on those.

The next big push will be when the 12/31 Accelerated filers start filing in earnest. Their deadline is 3/16 so that is not too far away.

As a reminder the dimensional flags will be really helpful so don’t forget those as you are working with this data.

We are continuing to test and refine our approach to capturing the impact of ASU 2023-09 on the structure of the Effective Tax Rate Reconciliation table. Our work is grounded in the actual table structures that companies have used to implement this expanded disclosure.

One of the key attributes we believe is important to capture is the adoption method (retrospective or prospective). In many filings, the adoption method is not explicitly stated. Instead, it must be inferred from the structure and content of the reconciliation tables. To address this, we developed an algorithm that analyzes table structure and determines the adoption method based on observable disclosure patterns. As a result, we have introduced a new field, ASU_STANDARD, to identify disclosures that conform to ASU 2023-09.

If the ASU_STANDARD field is blank, the associated rows were not disclosed in conformity with ASU 2023-09.

If the field reports ASU_2023_09_PROSPECTIVE, two implications follow for that accession number:

1. All rows labeled with ASU_2023_09_PROSPECTIVE represent the ASU 2023-09–mandated disclosure.
2. A second reconciliation table will typically exist for the same accession number that presents the Effective Tax Rate reconciliation using the prior disclosure format. This reflects the prospective adoption method, under which ASU 2023-09 applies only to the current period, while prior periods continue to be presented under the previous disclosure requirements.

As a practical matter, this assumes the filer is not a new registrant, since a newly reporting entity may lack prior-period operating history and therefore may not present a legacy-format reconciliation table.

If the ASU_STANDARD field has the value ASU_2023_09_RETROSPECTIVE, only one logical reconciliation table is expected for that ACCESSION, and that table will typically report up to three years of data.

The term “table” is used here in a logical rather than strictly physical sense. In many filings, the ASU 2023-09 reconciliation disclosure spans multiple pages in the Form 10-K and is therefore physically presented as two or three separate tables. However, these tables collectively represent a single reconciliation disclosure.

When this occurs, we treat the sequence as one logical table. All rows from the component physical tables are assigned the same TABLE_NO, which corresponds to the index of the first table in the sequence. This distinction between physical tables and logical tables reflects the presentation structure of SEC filings while preserving a consistent analytical unit for downstream processing. This approach ensures that the full multi-page disclosure can be reconstructed and analyzed as a single, continuous reconciliation table.

There is a structural issue I would like to address but I think it is too early to do so. The new standard requires separate disclosures by both the jurisdiction and nature of the tax/benefit if they are 5% or more of the total income taxes paid. The jurisdiction can be identified in two ways. First by the country name in the row label as you can see in the next image:

The problem with relying strictly on the row label is that the row label may be a caption for a subsection. To understand my meaning review these rows in the database from the 10-K filing submitted by TEVA PHARMACEUTICAL INDUSTRIES INC (CIK: 818686)

If you look closely you will see that none of the rows with the names of foreign jurisdictions have data. That is because those are section identifiers. To determine the country related tax effect it is necessary to use the dimensional information provided by the filer. When this information is disclosed we capture it in the explicit_member_dim (describes nature of the intended disclosure) and then the explicit_member_text field which provides the identifying information as you can see in the next image.

The structural issue I would like to address is to add country name fields with a value of 1/Null. However, I don’t think we have quite enough information yet to do so. We need a much larger sample of these disclosures to develop the right system to map the disclosures with the right country.

Speaking of a much larger sample, March 2 is the filing deadline for 12/31 Large Accelerated Filers. For those of you eager for this data to beat the crowd we are updating our data nightly and our client facing data weekly. However, we will make a special update early in the morning of March 3rd with the filings through 3/2. However, let me warn you that about 2-4% of the filers will be one or two days late. Further, the Accelerated Filers deadline follows on 3/16.

One of the promises of inline XBRL was simplicity.
If a data point appears on the cover page of a 10-K and it’s tagged, then—at least in theory—it should be easy to extract, compare, and analyze.

Public float is a good example.

The SEC requires filers to report their public float on the cover page, and with tagged cover pages that value is now explicitly labeled and machine-readable. Many researchers and data users understandably rely on the SEC’s structured data feeds to capture it.

But there is a quiet problem.

In a subset of filings, the tagged public float value does not match what the company actually reports on the cover page.

Here’s what that looks like in practice (Packaging Corp of America 12/31/2024 10-K):

• Reported on the HTML cover page
  • $16,124,636,098
• Tagged Value in XBRL:
  • 16,124,636,098,000,000

The widely used SEC Structured data files contain the same value as the tagged XBRL. To explore you can download the February 2025 data folder and after opening look at line 7,553 in the sub.tsv file.

That’s not a rounding issue or a scaling choice. It’s a conversion error—effectively inflating the reported public float by a factor of one million.

What’s especially interesting is that these filings share a common fingerprint. They appear to have been prepared using a specific filing platform, as indicated by metadata embedded directly in the submission document.

<!-- DFIN New ActiveDisclosure (SM) Inline XBRL Document -->
 note - creation date varied across the 30 so it was not a specific date issue
<!-- Copyright (c) 2025 Donnelley Financial Solutions, Inc. -->

Based on the above, this doesn’t look like a company-specific mistake. It looks like a systematic transformation issue introduced during the filing conversion process. However, it’s more complicated than that. Based on my analysis, this software was used to file 892 10-K (I excluded 10-K/A) in 2025 but the problem seems to have occurred in roughly 30 or so.

Why this matters

If you rely on the SEC’s structured data endpoint you’ll ingest the overstated value without any obvious warning. The number is valid XBRL; it parses cleanly. Unless you validate it against the rendered filing, it looks perfectly legitimate. Why would you question it?

That’s a problem for:

• researchers using filer size thresholds in their analysis
• analysts filtering by public float,
• and anyone using public float as a screening criteria

A simple safeguard

In our own workflows we actually start with the HTML filing itself and then map tagged XBRLvalues back to what the company actually reports on the cover page. This makes discrepancies easier to flag because even for our client facing databases we report to you both the original value as reported in the html and the tagged value as you can see in the next image

This imposes additional work but it also gives you a point of reference that is lost with the raw XBRL data no matter how you access it. Ingesting just the XBRL give no reference to truth. The Inline XBRL makes more data accessible, the HTML provides a clear way of evaluating the results.

Conclusion

When the first XBRL filings were introduced, we were ready—and genuinely excited. The promise was compelling: standardized, machine-readable financial statements that could be reliably constructed directly from tagged data.

That excitement didn’t last long.

As we began validating what we were deriving from the filings, it became clear that the work required was far more exhaustive than anticipated. In one early effort, we constructed roughly 5,000 income statements from XBRL data and compared those to the original financial statements. Approximately 7% contained problems.

What made this especially frustrating was not the error rate itself, but the nature of the errors. We were unable to devise any rule, heuristic, or algorithm that could reliably identify which statements were wrong. The issues were only visible when a constructed statement was compared to the one actually presented in the filing. So XBRL could not be a starting and ending point.

Inline XBRL has materially improved this situation. By embedding tagged data directly within the rendered document, it provides the necessary context to test, validate, and reconcile structured data against what filers actually report. That context doesn’t eliminate errors—but it makes them observable.

The lesson hasn’t changed: structured data creates opportunity, but only when it is paired with validation, traceability, and context. Inline XBRL doesn’t solve data quality problems on its own—but it finally gives us the tools to see them.

We are preparing to drop a fairly comprehensive Audit Fee data base. Long time coming but one of the real challenges is that there can be some wonky data and it is what it is. I am spending time trying to devise tests to evaluate the quality of our parsing and data assignment code. One of the tests I dreamed up is checking if the Date of the Audit Report precedes the dissemination data by more than 10 days. Here is a screenshot of the audit report for CSW Industrials (CIK 1624794) for the year ended 3/31/2022.

Here is a screenshot of the auditor’s signature:

It is not a huge issue, the filing was made on 5/18/2022 and perhaps someone felt harried when doing the final checks. For us though, it raises a significant question – when do we have liberty to change data after it is reported? I am so reluctant to make changes because that seems like a slippery slope. What is the source of truth? As an aside, the auditor submitted an Exhibit-18 describing the change from LIFO to FIFO that was filed with the 10-K. That has the date 5/18/2021.

We have identified about 100 different cases of similar/adjacent issues. Trying to validate that what we are doing is correct is our primary focus. A secondary issue though is to decide what to do. As I write this I am more inclined to use the 5/18/2021 date just because the 10-K should be the source of truth.

We just updated the TableExtraction and Dehydrator code to create new outputs.

For TableExtraction we added a new search_phrase.txt file that is only generated when you use the Search Query option to identify tables for extraction. The file contains both your original set of parameters as well as the transformation(s) we apply to generate the code that is then delivered to the TableExtraction engine. Here is a sample:

Input:  (acquired or price or assumed) and (goodwill  and intangible and asset and 
         liabil)
Parsed: AND(OR(Contains('acquired'), Contains('price'), Contains('assumed')),
        AND(Contains('goodwill'), Contains('intangible'), Contains('asset'),  
         Contains('liabil')))

The Input line reports the text you submitted and the Parsed line reports on how the line was transformed. What we hope is that this provides more visibility to you regarding why the results contained (or did not contain) particular tables.

We also redesigned the logging to create a new log file named SnipSummary_DATE_TIME.csv. This file contains all of the details from the input file and we have added a column named COUNT to report to you the number of snips that were extracted from each of the source files. Here is a screenshot of this file (I hid all of the metadata columns except for CIK).

The intention is to give you clearer visibility into the results. In the example above I was snipping purchase price allocations and their is not necessarily an upper limit on the number of those tables that might be reported in any particular 10-K. However, it was actually the case that for many of the snips above the snipped tables included the Statement of Cash Flows (as it included all of the strings I set as my parameter. I discovered that easily by initially reviewing the snips from those CIKs. There are many cases where you expect only one table from a filing – in that case you might see counts of 2/3/4 and you can identify those to review.

The MISSING.csv file also has been modified. In the last build the missing (the list of documents for which no tables were found) was a txt file. It is now a csv file. All of the metadata from the original file is present and there is an additional column DE_PATH. The reason for that column is so that you can then run a search to focus just on those documents. There is a demonstration of how to use the MISSING.csv file to run that search in this video (the search example begins at the 4’28” mark in the video). What is key here is that you can run the exact same search you ran initially, but the output is limited to only these specific documents.

Finally, another update was made to the Dehydration output. Malformed tables are now saved in a PROBLEM_TABLES subdirectory with an adjacent csv file that has the same name as the snip. The csv file contains all of the usual metadata from Dehydration/Rehydration and then we have parsed each line so that the content from the td/th elements ends up in a single cell. Here is a screenshot of this:

As you can see from that real example, this file will be easy to prepare for your data pipeline. You would just rename COL3, COL5 and COL7, delete COL2, COL4 and COL6 and then delete the two rows below (FISCAL 2022 DIRECTOR COMPENSATION and the row that has all of the original column headings).
Earlier, we stacked these in one csv file but after working with that some we believe this output is much easier to work with.

As a side note – after I saw the results with the SCF tables – I deleted all of the tables and output from that run and changed by Search Query to

Input:  (acquired or price or assumed) and (goodwill  and intangible and asset and 
         liabil) not (invest or financ or operat)

This modification reduced the noise in my output.  

We are frantically trying to wrap up a complete auditor/audit fee database. We need to plug a pretty significant hole with respect to the AUDITOR, TENURE, REPORT DATE and LOCATION for 10-K filings made in 2020. Why only that year, that is another story.

I ran a search for ((DOCTYPE contains(10k*)) AND date(1/1/2020- 12/31/2020) pre/5 we have served). In English, I am looking for cases where there is a date in the range of 1/1/2020 through 12/31/2020 where the date precedes the phrase we have served by no more than five words. Further, I wanted to limit my search to only 10-K or 10-K/A documents. (Note – most auditors report their tenure before the report date – for those searches we changed the order of the date and the we have served). Below is the results of the search where date precedes the the tenure.

I set a box around the Baltimore Gas & Electric Co. (BG&E) summary to highlight there were 55 matches in the document. There were 55 matches in that filing because that filing has nine cofilers (and the fact that a match is a bit more complicated.) Here is a screenshot of the context extraction from that search with a focus on the lines belonging to BG&E. As you can see Pricewaterhouse Coopers served as the auditor for each of the subsidiaries included (the parent company is Exelon Corp). However, the context does not identify the entity to which the block of text references.

As a researcher you have to decide if you want this data. If you need it then you will actually find 81 total rows in the context extraction results (9 10-K filings with 9 disclosures in each 10-K filing). Unfortunately, the only way to identify the correct disclosure per CIK is to dig back into the filing. It turns out that the disclosure that belongs to BG&E is the first one in the list above with the phrase since at least 1993.

I actually imagine in many cases that these particular disclosures are not important because you are likely to eliminate these types of filers from your sample. Again though, if they are important it is critical to review the duplicates carefully to match the data to the Central Index Key that it is associated with.

As a random aside – it is/was interesting to me to observe that most of the cases with this form of the disclosure (Auditor, location, date and then tenure) were found in filings with COFILERS. The more common form of the disclosure (as pictured below) had many fewer total COFILERS in the results.

I ran these searches separately to simplify the parsing of the data (AUDITOR, TENURE, LOCATION CITY, LOCATION STATE, REPORT DATE). from the context. I am lazy, I wanted more certainty about the order I was expecting the analysis to follow so the parsing code had less to test.

Much earlier in the week I posted about Excel auto-populating a new mapping based on the column headings found in a collection of tables that report the Equity Compensation Plan Information disclosure required of SEC Filers. Today, I am not frustrated but I do want to share my experience – particularly since I know some folks are all into AI.

I am still working on that table and I am reviewing the labels before we start discussing moving this table into production. We are too small to have a human compare the original tables with the mappings so this step has to be handled with significant care. We have 908 original column headings pulled from about 1,600 original tables. These column headings should generally map into three semantic concepts, the number of shares to be issued, the weighted-average exercise price of any options reflected in the number of shares and then the remaining number of shares available for future issuance. In about 2% of the cases there are four columns of data presented but the overwhelming structure is consistent with what is set out the CFR section which mandates this disclosure.

I wanted to do a test to accomplish two goals. First, I wanted an independent review of the mappings. Since I completed 909 rows I thought asking Manish to review it would be worse then drudgery. I also wanted to evaluate whether or not I could have saved the approximately four hours original mapping took. In other words, could I trust this tool for this task? I spent about an hour outlining the problem to a very popular AI tool and submitted the list of column headings. Below is a screenshot of the original labels as parsed from the disclosures

I then merged the output from that process with my original labels to identify any differences and identified the differences. Here is a screenshot of the cases where my mapping did not match the original mapping.

This is not horrible but it is concerning. My instructions described three main concepts – TO_BE_ISSUED, AVERAGE_EXERCISE_PRICE and REMAINING. I did not recall any heading that seemed to map into a concept of OUTSTANDING and I do not think any of the above are real examples of cases where the in this context OUTSTANDING is a better choice then TO_BE_ISSUED. The first two do have the word OUTSTANDING but I would argue with an editor about the word choice if I saw the word OUTSTANDING used in this context.

I asked the AI tool why and their response really seemed to be more of a justification then a valid answer. Here is an excerpt:

Step B — But the phrase also contains both a past and future state

“issued or to be issued”

This is a compound construction:

“issued” → past

“to be issued” → future

In equity-comp language, humans sometimes use:

“issued and outstanding” interchangeably (sloppily)

“issued or issuable” to refer to currently outstanding awards

The diagnostic logic (incorrectly, in hindsight) treated:

“issued OR to be issued”
as a state aggregate rather than a flow category

That aggregate was internally labeled OUTSTANDING — meaning:

“currently existing instruments, whether already issued or issuable”

Maybe an English major would draw the same conclusion – I’ll have to ask one.

Unfortunately, I did not accomplish what I set out to accomplish – which was an evaluation of my original labels. So I took a different tack and provided my mappings and asked my trusty intern (the AI tool) to evaluate the mappings. According to my intern – my mappings were perfect. I doubted that, any reader of this blog knows I am an expert error maker. So I went through the mappings line by line. Hugely tedious but I found two errors. One case where I mapped “OF SECURITIES REMAINING AVAILABLE FOR FUTURE ISSUANCE UNDER EQUITY COMPENSATION PLANS (EXCLUDING SECURITIES REFLECTED IN COLUMN )” to “TO_BE_ISSUED” and then one where I mapped a label that communicated information about the exercise price into REMAINING.

This is interesting to me because I am trying to figure out the value of AI integration somehow into our client facing tools. I understand there is no joy in doing the original mapping and then checking them. Wouldn’t it be hugely amazing to offload that work completely? I just don’t see it yet. I think we need to perhaps streamline (pre-map) the original labels with better cleaning. So maybe my 908 rows could be reduced to 300 or so. But if I have to stand by the results (in my case it is the quality of the data we deliver, in yours it might be a really interesting and novel research finding that you want to publish) I am not ready to have any of the existing tools take over data cleaning. They are a huge help in other ways but when it comes down to verifying the accuracy of data, I think we have some ways to go.

I will say there is probably some significant value in letting the tools do the heavy lifting. If in fact you can take the time to carefully explain the nature and features of the disclosures and you dump a significant list, then you are likely to save some initial time. But I would also think about how to audit the results.

We have been working to add some new data to the platform. Specifically the breakdown of the disclosure of the Equity Compensation Plan Information (EQCPI). I specifically have been evaluating the tests we need to identify the cases that need to be inspected. As good as our parser it we still get some bad parses and we need to be able to systematically identify those and shunt them to a review queue for a person to look at. For context, with Executive or Director Compensation, if the sum of the reported compensation differs from the reported total by more than $10,000 a human looks to see if the issue can be addressed (maybe a value is reported as 123.456 and on closer examination it appears that it should have been reported as 123,456).

What we are seeing in the EQCPI disclosures are errors in the disclosure that we can’t initially be certain of the source of the error. Here is what I saw in a summary file after I ran the code we are working on to identify potential errors.

This was reported in a proxy filing made by OXFORD INDUSTRIES (CIK: 75288). If I were collecting this data for research I would be concerned about the reliability of these values since there is no total for TO_BE_ISSUED. However, this is what was reported (other than the normalization choices we made) as you can see in the next image.

So while this merits review, the captured data accurately represents the disclosure that was made. For internal purposes though I need to be more confident that the data was correctly captured. I have decided that we should stop ignoring dashes and dash like characters and replacing them with something consistent. Currently, we replace dashes with nothing. I am leaning to start replacing them with something clever like wasDASH. I am also thinking about replacing blanks (the value for total under Number of Securities . . .) with wasBLANK.

The advantage of doing so is that we can reduce significantly the cases we have to review. If we do this internally, no big deal. However, I was thinking about whether or not this would benefit you when you are normalizing tabular data. Here is a screenshot of what I am considering.

In some ways this might seem excessive but I don’t know what I don’t know. My point is that to get it right we have to consider that something happened between extraction and dehydration. My lovely wife will tell you quickly that I am not perfect, the point I am trying to make is that some of these html source files can be in a shape and structure we have never seen before and so the error could be that one of our assumptions that is embedded into the code is wrong for this case.

The next time I send out an update I am going to survey you as to whether or not we should make this part of the normalization so that you see these values when the tables are rehydrated.