Invoice-in-a-box – 4 steps to success

Oct 04
2016

Invoices are one of the highest demanded documents to automate. Let’s talk a little about what it takes to be successful in invoice processing. Data Capture is the technology used for invoices. This is where you extract field-by-field the information you want from the invoice in field order. In order to automate invoices with the high accuracy and utilize a boxed invoice solution you need to do some preparation. Here are 4 MUST have steps:

1.)Separate your commercial invoices from any specialized invoice types such as legal, manufacturing, telecommunication, etc. The reason you do this is because the low hanging fruit when automating invoices is commercial invoices. Software packages have put the most amount of effort in these documents. By working with them first, you are ensuring your success on a large population of your invoices and then can tackle the remainder.

2.)Know how many vendors you have. Understanding the makeup of your invoices is very important. Your focus should be determined by those invoices that are easiest to automate and make up the greatest portion of your entire volume. So make a list of all your vendors and what paper volume percentage each makes up of the whole.

3.)Know if you want to collect line-item data or not. At first glance, majority of companies say they want line-items, only later to change their mind. Find that business process that mandates you collect line items. In your current process, are you having line items entered? What database of existing information will you use to support your line-item extraction? Most companies in the end choose against line-items or choose to extract them for limited critical vendors.

4.)Know how you are going to check the quality of extraction. Quality assurance happens with human review, and business rules. Know before hand how you want those to work. For example a business rule simply could be all line-items must add up to total amount, if they don’t you have someone, look at the entire invoice.

These four steps are not the end-all in proving you invoice processing accuracy, but they are necessary and all steps to consider before you look and purchasing a boxed invoice processing solution.

Chris Riley – About

Find much more about document technologies at www.cvisiontech.com.

Even OCR needs a helping hand – Quality Assurance

Aug 04
2016

Let’s face it. OCR is not 100% accurate 100% of the time. Accuracy is highly dependent on document type, quality of scan, and document makeup. The reason OCR is so powerful is because it’s not. How do we give OCR the best chance to succeed? There are many ways, what I would like to talk about now is quality assurance.

Quality assurance is usually the final step in any OCR process where a human reviews uncertainties, and business rules based on the OCR result. An uncertainty is a character that the software flags that did not during recognition satisfy a threshold. This process is a balancing act between a desire to limit as much human time as possible and a need to see every possible error but not more.

Starting with review of uncertainties. Here an operator will look at just those characters, words, sentences, that are uncertain. This is determined by the OCR product which will have some indicator of what they are. In full page OCR, often spell checking is used. In Data Capture, usually a review character-by-character of a field is done and you don’t see the rest of the results. Some organizations will set critical fields to be reviewed always no matter the accuracy. Others may decide that a field is useful but does not need to be 100%. Each package has its own variation of “verification mode”. It’s important to know their settings and the levels of uncertainty your documents are showing to plan your quality assurance.

After the characters and words have been checked in Data Capture, there is an additional step in quality assurance, business rules. In this process, the software will apply arbitrary rules the organization creates and check them against the fields, a good example might be “don’t enter anyone in the system who’s birth year is earlier than 1984”. If such a document is found, it is flagged for an operator to check. These rules can be endless and packages today make it very easy to create custom rules. The goal would be to first deploy business rules you have already in place in the manual operation and augment it with rules to enhance accuracy based on the raw OCR results you are seeing.

In some more advanced integrations, the use of a database or body of knowledge is deployed as first round quality assurance that is also still automated.

These two quality assurance steps combined should give any company a chance to achieve the accuracy they are seeking. Companies who fail to recognize or plan for this step are usually the ones that have the biggest challenges using OCR and Data Capture technology.

Chris Riley – About

Find much more about document technologies at www.cvisiontech.com.

Capture Products, Data Capture Products, confused?

Jun 16
2016

All technology markets are guilty of coming up with at least one or two confusing terms. In the document imaging world, it’s terms with very similar sounding names. They are technically similar, but strictly different.

One of the most confusing things in the imaging world is the difference between Image Capture software often just called Capture, and Data Capture software. Not only are the names confusing, but technically there is a lot of overlap. All data capture products have imaging capabilities, all capture products have basic data capture. The risk of the confusion is replacing one product for the other. For example, organizations that attempt to take the data capture functionality built into a capture application for a full blown project, end with little success and a lot of frustration. Let me explain where they fit.

Capture products have the primary function of delivering quality images in a proper document structure. They often feature image clean-up, review, and page splitting tools that are more advanced then the scanning found in data capture applications. Most demonstrate what is called rubber-band OCR, the reading of a specific coordinate on a page. Some go as far as creating templates where coordinates zones are saved. This is where the solutions get confused with data capture. Until there is a registration of documents and proper forms processing approaches, it is not data capture. The risk of such basic templates is low accuracy and zones that do not always collect data.

Data capture products need images to function, so it was an obvious choice to add scanning to the solutions. These solutions however are better fed by a full capture application that has the performance and additional features such as batch naming, annotations, page splitting, etc. that the organization may require in the resulting image files. For data capture, the purpose of image capture is for getting data only and sometimes neglect the features that are important for image storage and archival.

In the end, both solutions are improving in the other’s territory. Eventually the lines will blur to the point where feature-wise they will be identical, and the benefit of one over the other will be rooted in the vendors expertise, either capture or data capture. If your primary requirement is quality images, the capture vendors solution is best chosen, but if it’s data extraction, then data capture rooted solutions are better.

Chris Riley – About

Find much more about document technologies at www.cvisiontech.com.

Outsourcing document recognition

Apr 28
2016

It’s common for organizations to outsource their scanning and document conversion. Organizations find it sometimes that the skill required, the convince factor, and liability is worth the additional cost. Other organizations that have one time backlog conversions save money by using an outsourcing company vs. bringing the software in-house. In recent years, service bureaus and business process outsourcing companies have dramatically improved their use of recognition technology and prices have dropped substantially. Though as an organization who chooses to outsource, you are removing the responsibility of picking document conversion technology. Shouldn’t you want to know what technology your service bureau is using?

YOU SHOULD! Absolutely you should be concerned about the OCR and Data Capture technology that your outsourcing company is using. It’s just as important than if you were bringing the technology in-house. It’s your job to make sure your vendor is using the best technology but also in the best way. The education level between outsourcing companies is different and they each often specialize in one document type or one type of processing. Proper evaluation of a service bureau will include reviews of sample results. You should have your prospect service bureau or BPO run a good number of your production documents and provide you with results. Make sure the technology they used to produce the results is the same that is used when in production. Don’t be afraid to ask the vendor what engine or engines are being used and even what version. Make sure you understand how your vendor handles exceptions.

While it’s easy to overlook these items when you are looking at a service instead of a technology, it’s still important that you are educated. Service bureaus make money based on how much they save. This can occasionally create motives to use poor technology to gain greater margins. Some outsourcing companies put customers into categories by volume and those with the greater volume get the best technology. Most of the outsourcing companies out there are very good at ensuring their document quality, and many will even go as far to give you a guarantee on quality. But the nature of production environments is such that you cannot check everything always. It’s about relationship. Sometimes paying a higher price per page for a better solution is worth it!

Chris Riley – About

Find much more about document technologies at www.cvisiontech.com.

Line-Items : Picking the correct field type

Feb 22
2016

Documents containing tables have the majority of information of the document printed thus the demand to collect this data is very high. In data capture organizations will choose three scenarios to collect data from these documents; ignore the table, get the header and footer and just a portion of table, or get it all. Ideally organizations prefer the last option, but there are some strategic decisions that have to be made prior to any integration using tables. One of those decisions is whether to capture the data in the table as a large body of individual fields or as a single table block. Lets explore the benefits and downside to both.

Why would you ever perform data capture of a table with a large collection of individual fields when you can collect it as a single table field? Accuracy. Theoretically it will always be more accurate to collect every cell of a table as it’s own individual field. The reason for this is because you will accurately located field, remove risk of partially collected cells or cells where the base line is cut, and remove white space or lines from fields. In some data capture solutions this is your only choice. Because of this many have made it very easy to duplicate fields and make small changes so the time it takes to create so many fields is faster. This is a great tool because the downside to tables as a collection of individual fields is in the time it takes to create all fields and maybe this is too great to justify the increase in accuracy.

If you have the ability in your data capture application to collect data as an individual table block, you are able to very quickly do the setup for any one document type. Table blocks require document analysis that can identify table structures in a document. The table block relies heavily on identified tables and then applies column names per the logic in your definition. This is what creates its simplicity but also its problems. Sometimes document analysis finds tables incorrectly, more often partially. This can cause missing columns, missing rows, and the worse case scenario rows where the text is split vertically between two cells or horizontally cutting columns in half.

There is a varying complexity in the tables out there, and this most often is the deciding factor of which approach to take. Also very often the accuracy required, and the amount of integration time to obtain that accuracy determines the approach. For organizations that want line-items, but they are not required, table blocks are ideal. For organizations needing high accuracy and processing high volume, individual fields are ideal. In any case, it’s something that needs to be decided prior to any integration work.

Chris Riley – About

Find much more about document technologies at www.cvisiontech.com.

Barcodes, time savers, and wasters

Feb 17
2016

Barcodes are a great technology. You can fit a lot of information in a barcode, they can be read at any angle, and they are also very accurate. You have to degrade 30% of a barcode before it’s unreadable. In data capture, barcodes are commonly used for batch cover sheets, document separation, or printed on the document themselves. This has been proven to be a time saver both in quality and because they can be read very quickly using both software based and hardware based solutions. What organizations often don’t think about is the additional time and cost that barcodes add to the capture process.

Organizations usually don’t connect document creation and prep time with data capture time. The total time and cost associated with the capture of documents is not just from the point of scan to export. It is all the additional steps leading up to the scan to get the document in the state it needs to be fore scanning. If an organization uses barcode pages to separate documents, it’s the time it takes for an operator to generate the pages and put them manually between documents. If organizations use barcode pages as batch separation, it’s the time it takes to create the unique barcode for each batch and place it on top of the batch prior to scan. These are just the two most common examples but there are many more.  This is a common misconception because it’s not the same person doing the barcode creation and separation as the person scanning, or the barcodes are created in advanced and the time it took is forgotten.

Because organizations are not counting this into the total capture process they are missing out in the real data capture time and cost. It’s no surprise then when they are maintaining high paper cost and not reaching the ROI they expected. Barcodes are a great tool, but should be used when their benefit is greater then their time cost. Benefits can be accuracy, and process molding. Very seldom are barcodes alone responsible for substantial cost savings. Very often organizations don’t realize that they could in fact do away with barcodes by using advanced data capture. Accuracy may surfer slightly but the time savings is substantially more.

Chris Riley – About

Find much more about document technologies at www.cvisiontech.com.

Not quite as fun as the DMV

Jan 25
2016

Understanding the different licensing that is available for data capture and OCR products can sometimes be difficult, but I assure you that the complexities involved will not be as painful as a trip to your local motor vehicle. There are a few aspects of licenses that trip up some users namely license type dongle or serial number, activation process, and finally page-counts.

License type can be very important but is not often clearly explained. The most common license type out there is “software license”. This is a license structure that is a license file tied to a specific machine. The benefits of such a license are, it’s more efficient and easier to install on servers and hardware that are not local. The downside is that because it is tied to a machine, if the license dies you may have downtime while waiting for replacement and proving destruction or may have to purchase a new licenses. Another very common license type is a hardware dongle. Dongles now are most often USB devices very similar to a USB thumb drive we are all used too. The benefit to this type of license is that the software can be installed on every machine in the organization but only the machine with the dongle in can run it. This means that if something happen to one machine it would be very easy to switch to another. The downside to this type of license is that the licenses can be lost, and it’s not the most efficient. After you have whatever license type it is, you will need to go through the activation processes.

Activation can be troublesome for some products and others very simple. The difference is usually the installers effort in understanding the activation processes BEFORE any installation. For many of these products activation has as many as 3 steps and it’s usually always in the form of sending an activation request, receiving an activation file, installing the activation file. The trend is for products to allow web activation and it’s becoming more popular, but because of the premium on some advance data capture products these steps are required. Now with an activated license the most important thing, what does a license give you?

Licenses are usually set with general operation rights, purchased add-on’s if they exist, and very commonly page-count. Page-count is the biggest contention of most any purchaser. Because of this most all vendors have the option to have unlimited page-count license for a premium. In the end most all companies end-up with a page-count licenses and are quite happy. What argument I would like to pose is that a piece of hardware has inherently a page-count, as each piece of hardware will only be able to physically process a certain number of pages a day, month, year. For this reason page-count is actually quite reasonable but a slowly dieing trend. In the future I expect to see far fewer page-count licenses. For most businesses pages are counted on a monthly basis but some seasonal companies may elect for an annual or pure page count.

License structure is important to ALL organizations and I encourage companies to spend the time during the discovery phases of technology acquisition to investigate the structures that are available from each vendor and how that may work in your environment.

Chris Riley – About

Find much more about document technologies at www.cvisiontech.com.

Fixed, Semi-structured, UNSTRUCTURED!?

Jan 13
2016

I find myself educating even industry peers on the topic of document type structure more and more recently. Often the conversation starts with one of them telling me about how unstructured document processing exists, OR the fact that a particular form is fixed when it is not. Understanding what is meant when talking about document structure is very important.

First lets start with defining a document.  A document is a collection of one or many pages that has a business process associated with it. Documents of a single type can vary in length but the content contained within or the possibility of it existing is constrained. When data capture technology works, it works on pages, so each page of a document is processed as a separate entity and this it seems, is the meat of the confusion.

Often someone will say a document is unstructured. What they are thinking of is that the order of pages is unstructured, this is more or less accurate, however the pages within this unstructured document are either fixed or semi-structured. The only truly unstructured documents that exist are contracts and agreements. How you know this is that if at any moment in time you pull a page from the document and state what that page is and what information it would have, then it IS NOT unstructured.

The ability to process agreements and contracts is very limited in very concrete scenarios, where the contract variants are non-existent which essentially also makes them unstructured. In general the ability to process unstructured documents does not exist. Now to explore the difference between semi-structured and fixed.

It’s actually very easy because 80% of the documents that exist are semi-structured. Even if a field appears in the same general location on every page of a particular type, it does not make it fixed. For example, a tax form always has the same general location to print the company name. The printer has to print within a specified range. They can print more to the left, more to the top, and the length will very with every input name. This makes it semi-structured and additionally this document when it is scanned will shift left , right, up, down small amounts. A document is ONLY truly a fixed form when it has registration marks and fields of fixed location and length. Registration marks are how the software matches every image to the same set of coordinates making it more or less identical to the template.

There again the confusion is exposed. It’s very important to understand when having conversations about data capture to understand the true definitions of the lingo that is used. I task you, if you catch someone using the lingo incorrectly, it will help you and them to correct it.

Chris Riley – About

Find much more about document technologies at www.cvisiontech.com.

Guarantees, Guarantees, Guarantees

Jul 14
2015

One of the most popular questions to ask when organizations purchase data capture or OCR software, “what accuracy can you guarantee?”. If you have ever asked this question of a vendor you got one of two responses: the first was a percentage of accuracy, the second is a long explanation on why they can’t guarantee anything. If the vendor gave you a percentage you should probably run, because it’s the start of a bad relationship.

Why? It’ not really possible for a vendor to tell you how accurate your recognition will be on your documents. Vendors can estimate accuracy based on samples, they can give you an idea of range, but because of the nature of the technology there is no way to guarantee anything. The first fact of OCR is that you can ALWAYS find a document that breaks the norms of recognition and accuracy. Because of this possibility, it’s hard to know how exception documents will effect the accuracy of the entire system. So lets talk about what is reasonable.

It is reasonable to provide a sample set of documents and expect an average accuracy level as a percentage on the samples. Because they are a discrete subset of documents, this is something that can actually be measured. It is the job of the organization to pick samples that most closely represent production. It would be wise to include bad, average, and good documents in the sample set so as to cover the entire range of possibilities.

What organizations often forget is that even if 50% of the documents are automated there is a cost savings as compared to manual entry. The industry standard for accuracy is 85% however this changes heavily based on document type and the organizations perception of accuracy. The ideal way to measure accuracy is to compare recognition results to truth data. If truth data is not available the next best thing is to count not accuracy but level of uncertainty on the document. If a document is 5% uncertain according to the OCR engine, then it is 95% certain and this should be your measure.

Next time a vendor is faced with the question of “how accurate are you?” or “what accuracy do you guarantee” I hope they issue the proper response of “how accurate will your process allow us to be?”. It’s a fair question to ask when you are not familiar with the technology, but hopefully the above gives you the proper approach to measuring a solution.

Chris Riley – About

Find much more about document technologies at www.cvisiontech.com.

Clock is ticking

Jun 30
2015

When considering the ROI on a data capture integration, setup time is one of the most important and often miscalculated factors. Not just the setup time for initial integration, but the setup time used for any fine-tuning and optimization may sometimes postpone production.

The difference in setup time between a fixed data capture environment where coordinate based fields are used and rules based semi-structured environments is substantial. It’s not usually the fixed data capture environments that pose the biggest challenge in calculating ROI or predicting it. It takes an administrator on average between 15 to 45 seconds to create and fine-tune a fixed form field. In semi-structured processing, the field setup time can take between 60 seconds and hours, depending on the complexity of the document and the logic being deployed. It’s this large gap that throws a wrench in some ROI calculations.

For experienced integrators, the ability to put a document and it’s associated fields into complexity classes is usually pretty easy. After doing so gauging, the average amount of time to setup each field, and thus all fields should be accurate. There is always a field or two that requires extra fine-tuning. The key is a complete understanding of the document. Sometimes document variations are obvious, other times they sneak up on you and you have no idea the variation exists until you start working with it. Knowing all variations is the easiest way to understand the additional time any field will take to setup. Variants are the biggest contributor of time in semi-structured data capture setup. Second is odd field types, such as fields that take up one to many lines, or are continuous across two separate lines, and finally tables. The third and final largest contributor to setup time is poor document quality. This means the administrator has to be more general when creating fields and likely has to deploy multiple logic per each field to locate information in several possible ways.

When calculating the ROI on your data capture project, make sure to be aware of these sometimes sneaky factors that can eat at integration time. Bottom-line, know your documents, and know the technology before any work is done. If you are unsure, seek professional assistance.

Chris Riley – About

Find much more about document technologies at www.cvisiontech.com.