Interested in an alternative to Kofax?

deltaecho · 2009-12-14 16:48:59

I've begun developing an application that will serve as an open source alternative to those provided by Kofax, Abbyy, and Iris. Once completed, it will support plugins, be able to generate, print, and read barcoded cover sheets, import documents, capture documents from a scanner, extract text from documents, and export them to an external file (or in my case, a hot folder mapped to an Alfresco repository).

It was started out of necessity, as many of our clients are smaller businesses who cannot afford to drop $10 grand on a scanning station, and should they choose the more affordable desktop application, would prefer not to pay an additional $1.5+ grand for the privileges of generating and scanning barcoded cover sheets.

I have no intention of developing my own barcode or scanning libraries, or engineering my own OCR engine, but instead have chosen to leverage existing open source solutions that accomplish those tasks, and have gotten the application off to a good start.

Is there an interest in the community for such an application? To my knowledge, there is no [working] open source solution of this type in the wild, so it would be the first of its kind. If there is an interest, under what license should I place it? And are there any developers who would be interested in helping me get it off the ground?

The core application is written in Java, as I am most familiar with it and Java is cross platform (I can compile a single binary on 64-bit Linux and deploy it on 32- and 64-bit Linux, Windows, [Open-]Solaris, BSD, Mac OS-X, etc.), but there is plenty of room for C++ developers as interacting with the OCR engine and the Sane and Twain drivers requires use of the JNI (Java Native Interface), which allows the interoperability of Java and native C/C++ code.

Any help or suggestions I can get will be greatly appreciated. I am currently the only developer working on this, and could really use some assistance by somebody with a bit more experience than myself.

deltaecho · 2009-12-15 22:02:15

Is there no interest in such an application?

How about this: under what license should I place the application? There is the obvious choice of the GPL, but even though I am developing the application entirely using free software, it will have some proprietary extensions -- as is the case, would the GPL (v2 or v3) be a good choice? What about the Apache license? I have no experience dealing with application licensing, and need some help in this area.

If I can settle on a license, I will go ahead and upload the code to SourceForge.net.

deltaecho · 2009-12-16 14:17:16

Not even a discouraging word from Allan?

Last edited by deltaecho (2009-12-16 14:18:14)

firecat53 · 2009-12-16 17:13:27

I have a friend who pretty much only keeps windows because he scans and organizes all his documents with a similar program....perhaps this would be what he's looking for.

Good luck to you!!

lswest · 2009-12-16 18:38:12

I can think of a few people who would be extremely pleased by such a tool, maybe enough so as to make the switch to Linux that's been hampered by the lack of such a program before. So I'd say there is definitely interest in such a program. However, I can't help too much with the licensing, since I'm unsure of the exact terms for any of them. I would imagine the GPL would be suitable though, except that the proprietary segments may need to retain their original licenses, that's all.

Either way, good luck to you! If I think of any way I could be of service, I'll drop a message here.
Lswest

deltaecho · 2009-12-16 19:59:31

@firecat53 and @lswest, you are the best! Thank you for your kind comments.

Regarding the libraries I'm using, here's the rundown:
1) Reading barcodes: http://code.google.com/p/zxing/
2) Generating barcodes: http://barcode4j.sourceforge.net/
3) Image processing: http://rsbweb.nih.gov/ij/
4) Reading PDF files: https://pdf-renderer.dev.java.net/ (I will probably switch to jPod: http://opensource.intarsys.de/home/en/i … urce.JPod)
5) Writing PDF files: http://itextpdf.com/
6) OCR: http://code.google.com/p/tesseract-ocr/ (and, eventually: http://code.google.com/p/ocropus/)
7) TWAIN/SANE driver abstraction: http://www.mms-computing.co.uk/index.php

Those are the heavy-hitters, there are still a number of other libraries I'm using. Right now, I'm working on engineering a [simple] database for handling plugin dependencies, and am considering eventually switching to Berkeley DB.

skottish · 2009-12-17 03:10:46

I think that this is great and can see any number of cost challenged or cost conscious businesses being interested in this. As you know, companies routinely spend a fortune on software that isn't particularly unique or excellent. If you, or anyone else really, can get in the door with a solution like this, it can be mutually beneficial. I wish you the best of luck with this.

deltaecho · 2009-12-17 14:33:59

Thank you @skottish, your words are very encouraging, and welcome any time.

Has anybody dealt with Java's ProcessBuilder class? I've run into an issue where if the JVM does not allocate enough memory to the application, it won't run correctly (strange -- ), and would like to avoid having to write custom scripts for every environment the application runs on. I would like to create an initial -- maybe bootstrap-ish -- class that will initialize the application in a new process with the required resources; I tried using Runtime.exec(...), but it resulted in some unexpected side effects (such as not theming the application's widgets correctly).

If I initialize the application in a separate process, is there any way I can execute it separately from the bootstrap one? To conserve resources, I would like to original process to terminate after the application is initialized, which would mean the application's process would have to run independently from the bootstrap's.

Edit: Cool! I worked around the issue with a JavaScript and the Rhino JS parser!

Last edited by deltaecho (2009-12-17 17:50:31)

kaismh · 2009-12-17 23:27:31

Hello deltaecho, actually this topic is of great interest to me. An open source capture module such as Kofax will be great. I have years experience in Document Management Systems (DMS), both programming and design. And I have used Alfresco and other CMs. People interested in such solutions are usually big organizations with loads of money. That is why Kofax, FileNet and others can charge the ridiculous amount of money.

I am currently doing consulting work for a local company in Jordan specialized in DMS and capture solutions, few years ago I was also working as a developer in their DMS team. I do encourage them to use more open source tools in their in their products. Nevertheless, there is a big gap between commercial imaging tools and open source ones. Omni OCR, Pixtran and ISIS.

I don't think Java is the way to go for such a project for various reasons. C++/Qt could be a better solution.

Also you need to think carefully about your target users. If they are big organizations support is a must.

Since there is good open source CM such as alfresco, storing the documents is not a issue. You can communicate with them via webservices.

I would like to help as much as I can. I do believe that spending some time on the design and plan is very important before rushing into coding.

deltaecho · 2009-12-18 01:17:50

@kaismh, thank you for your reply!

I agree wholeheartedly with you about preferring C++/Qt to Java for this project, and that it would be best to spend time engineering the application before implementing it, but I have only a little experience programming in C++, and my employer wants to deploy a beta application to our clients within a couple weeks. Being the only developer working for him, as an intern with only a little more than two years of experience writing software, my resources and abilities are limited.

Java is the official language of my university, and as such, I am a competent Java developer. I am also fairly apt with JavaScript, Bash, Python, and XML-based languages such as XHTML, XML Schema, and XSLT, but have not done much with C++. As is the case, how would you suggest I proceed? Should I learn Qt before continuing, and rewrite what I've done in C++?

kaismh · 2009-12-18 11:12:18

@deltaecho

The core syntax between Java and C++ is similar, although it will take some time to get familiar with C++ libraries and the Qt framework. I understand your employer situation, as they need to show the client a working demo soon. But they need to look at the big picture. Any open source project is as good as its community. Java in the open source application land is not popular, and I am not sure if it will attract many developers in the near future.

Also, for your application C++ will be a much better choice, speed, native connectivity to existing imaging libraries, .... You will need to be thinking about the following issues.

1) Target OS: if you are targeting all big 3 (Linux, Mac and Windows) then both Java and Qt will do the job. If you are only targeting Linux, why not build it as a KDE application, I understand that KDE libraries provide a great deal of functionality. Have a look at http://kooka.kde.org/, it is a KDE 3 application, not sure if has been ported to KDE4 yet. Last I checked KDE for both Mac and Windows was still alpha/beta.

2) UI: I would imagine that you are trying to build a user friendly interface, Qt designer is a great and easy tool for that. I have not done much programming in Java, but last I checked there were different and not so powerful toolkits.

3) Community: you will need to think about attracting both users and developers to your project.

4) License: I would think GPL will be a better option than LGPL and less risky. Alfresco is GPL right?

5) Business Plan: How are you going to make money out of it? and keep your employer happy.

deltaecho · 2009-12-18 19:31:03

I am discussing your comments with my employer now, they are very good questions.

If I were to switch to C++, what libraries could I substitute for what I already have? Also, regarding the GPL, would it allow me to develop application extensions to interact with proprietary software? That is one of my primary concerns at the moment, because the application will have to be able to communicate with some closed source applications.

Our targeted audience is primarily users of Linux and Windows, and ensuring the application runs well on Linux is of utmost importance to me. As is the case, I will probably not make it a KDE application, even though I love the desktop environment. Once KDE is well supported on Windows, however, I may likely change my mind.

I didn't know about Kooka, its development seems to have stopped approximately the same time I started using Linux, so it was before my time -- it looks like it was a very good application, though, and I may be interested in reviving it.

Java has two official toolkits, AWT and Swing, with the former being all but replaced by the latter (few developers still use AWT). For a little while, Qt was sponsoring the development of some Java bindings for their toolkit, Qt-Jambi, but its support was ended when Qt 4.6 was released. I believe the Eclipse Foundation is also sponsoring the development of their own toolkit, SWT, but I'm not very familiar with that one.

We currently plan on implementing a service-oriented business model around the application, but have not decided on all the specifics yet.

kaismh · 2009-12-19 00:12:38

-- what libraries could I substitute for what I already have?
I am not sure what libraries are you refering to? The Java or open source imaging (OCR ...)

About the GPL from my basic understanding that you can access/interact with proprietary software from GPL software, but not the other way around. There is GPL application for Windows interfacing with the proprietary Win32 API. I am not an expert on the subject, you will need to ask/read more on the subject.

I agree with you on not using KDE libs on Windows for now, they need to mature more first before using them. Sticking with Qt libs is a better option at this stage.

I suggest that you use a agile method for development. Start with a simple app that scans and save the douments in Alfresco. Afterwise, start adding one feature at a time.

If you have a solid simple application, which does what its supposed to do, you will attract intrest from the community.

deltaecho · 2009-12-19 01:25:55

I think you're right, as long as the extension is not dynamically linked with the proprietary application:
http://www.gnu.org/licenses/gpl-faq.html#GPLAndPlugins

The application will only need to query proprietary databases, so no linking will be required. I'm waiting to see how my boss feels about the GPL, but I believe it is the way to go.

Regarding alternative libraries, I would need something to read and write barcodes and PDFs, and interact with the TWAIN and SANE drivers -- are you familiar with anything? Most of my time the last couple weeks has been spent researching into what [Java] libraries are available for our needs, and have chosen the ones mentioned above. I'm sure there are C++ libraries that accomplish the tasks, it is just a matter of tracking them down (or developing our own, but that would take a lot of time and research).

The application doesn't scan documents yet, but it does have basic support for plugins, can read and write barcodes, and import documents; I'm thinking about switching from the PDF-Renderer library to jPod because PDF-Renderer only supports up to version 1.4 of the PDF spec, and cannot read newer PDFs. Currently, I'm working on getting the application to export documents correctly, but I don't think that will take much longer.

My plan is to implement scanning and OCR capabilities after we demo the application to our clients. Until then, I want to work on making the application function as solidly as possible with its current features -- does that sound like a good plan? I will upload the application to SourceForge.net as soon as I get confirmation on a license from my boss.

How do you feel about the name Socr3 (pronounced, "Soccer", a.k.a. football)? It stands for Open Source Optical Character Recognition, Reading, Rendering, and Exporting. It doesn't sound very professional, but is the best name I've come up with so far.

deltaecho · 2009-12-20 21:34:33

May I present, IAmNotKateTheTextEditor: http://www.mediafire.com/?zjbzwjh2o2n
(It is just a simple text editor, you don't have to download it)

Who says I don't learn quickly? Anyway, Qt is Qool and was very easy to pick up -- as was Doxygen; now I'm going to play around with the JNI and study a little into combinations and permutations. I don't think it will be difficult at all to port the application to Qt and drop its Java core.

I'm going to continue developing the Java version until I get permission to port it to Qt. Also, until I find C++ substitutes for the libraries I'm already using, I can continue using them through the JNI.

kaismh · 2009-12-22 04:58:58

Glad to hear you are learning Qt quickly and enjoying it.

I found the name Socr3 too geeky/technical. It is better to go with a simpler name for the sake of end users. I think the name should focus on the fact it is open and it does scan papers.

For example:

OpenScan, OpenDoc, OpenPaper, OpenCapture, .....

deltaecho · 2009-12-23 15:55:29

Ok, I will have to think more about the name. OpenCapture sounds good, but there was a similar project that was started and abandoned about a year ago with the same name (it looks like it didn't get much publicity, and its developers became discouraged); if I can track down its developers, they might be willing to help and I wouldn't have a problem, then, using the name OpenCapture.

For now (until I decide on a name), I've uploaded the source code to my MediaFire account -- here is a direct link: http://www.mediafire.com/?e40ttmzmngi

That will download a file, Socr3-Installer.jar, that if executed, will create two directories: $HOME/.Socr3 and $HOME/Desktop/Socr3. The .hidden file contains the application's libraries, config files, logs, and two plug ins, and desktop folder contains the Socr3.jar application archive, source code, and javadoc documentation. To tweak the application, you may modify the $HOME/.Socr3/init.js file, and to format the output from Log4J, $HOME/.Socr3/config/log4j.properties. The application has two log files, one may be found in $HOME/.Socr3/log/socr3.log, and the other is only created if Log4J cannot be instantiated, which is $HOME/.Socr3/fatal.log.

You don't have to execute the installer, though; since JAR files are just ZIP files with a different extension, you may simply unzip the file to get the source code.

Currently, for importing documents, the application is slow and a resource hog, and I wouldn't suggest checking out that feature unless you have about a gig of memory to dedicate to the application (~700 MB should be fine, but if you want to open a large PDF, I strongly suggest a gig to keep the application [and your system] from becoming unresponsive). The init.js file allocates a maximum of 1024 MB by default, but the application should run fine on 100 MB or less if you don't want to import documents (the JVM will only use that much RAM if it is required). Don't try to export documents because that feature doesn't work correctly yet.

Regarding barcoded coversheets, the application generates, prints, and reads them well.

Also, remember, this is an early release and VERY beta, or late alpha-ish, and is lacking several features. Let me know if the installer doesn't work correctly, or if the application fails to initialize.

pigbarve · 2009-12-29 15:18:17

Hi deltaecho,

There is not a software like this in the opensource's market, congratulations for the initiative.
Kofax is the leading software in document capturement, and I was looking for a choice by years.
The feature about alfresco integration is so cool.
It would be interesting to get a tool to define a template for scanning documents and getting specifics fields from it.
Reading fax, e-mails and another sources of data, and sending this to the repository would be appreciate.

I have installed the Socr3-Installer.jar, and if works fine.

Thanks and Good luck to you!!

deltaecho · 2010-01-01 16:35:47

Thank you @pigbarve! I am working on creating barcode templates now, and will work on the other features you suggested soon. I have not yet experimented much with the available open source OCR engines, but I hope I can use one of them to grab specific fields from scanned documents.

I've just pushed another update:
http://www.mediafire.com/?jnw1bxgyn1l

The only new features I introduced with this update are the ability to export documents and control the quality of images rendered in the previewer; I spent most of my time refactoring and improving the application's efficiency -- it is much more efficient and less resource intensive now (it's funny what happens when you initialize 100+ threads simultaneously).

PDF-Renderer has been replaced with jPod for reading PDFs, and I would like to replace iText with jPod for exporting PDFs as well, but ran into some issues when I tried to create pages with it; so, for now, both the jPod and iText libraries are included with the installer, which has increased its size by about 5 MB. Be assured, though, I will continue playing around with jPod until I can drop iText as a dependency, or until I learn how to parse PDF pages into displayable images with iText so I can drop jPod as a dependency.

Edit: 01/01/2010

Also, I have tracked down and begun talking with the OpenCapture devs about possibly merging our applications, so I hope soon to have more developers working with me.

Last edited by deltaecho (2010-01-01 16:38:35)

Heller_Barde · 2010-01-06 10:25:01

It would be cool if you could put the config files in $XDG_CONFIG (usually it's ~/.config) because as we actually have a standard for that, it's a shame if new software doesn't use it.

Cheerio
Phil

deltaecho · 2010-02-14 20:03:10

I'm still working on the application, but school started shortly after my last upload, and it has since consumed most of my time. I will upload a new version soon, and will implement your suggestion, @Heller_Barde, so stay tuned.

Other than your typical optimizations and bug fixes, the new version also has preliminary barcode templating and watches directories for files to process. My multi-threading skills are continuously improving, and I've changed many of the old algorithms to reflect my new abilities, so the application is more responsive and less resource intensive.

I'm trying to find a good algorithm for reducing the memory required to store all the buffered images, without annihilating the application's performance -- does anybody know of any?

Also, after my next upload, I will check into porting the application to Qt, but may remain with Java since Java 7 (i.e. version 1.7) will be based entirely on OpenJDK and will be much faster and lighter. It will also replace the current cross-platform theme, Metal, with another, Nimbus. If I were to switch to Qt, would anybody be willing to contribute? That is one of my biggest deciding factors regarding what toolkit to use, since I can develop it by myself, but would be much happier to have other developers to work with.

pigbarve · 2011-01-15 16:18:52

Hi deltaecho,

Just for sending greetings and know how the project is going on?
Did you continue with Java? (I Hope ;-)) or did you move to QT? (Also well)

Best regards

Arch Linux

#1 2009-12-14 16:48:59

Interested in an alternative to Kofax?

#2 2009-12-15 22:02:15

Re: Interested in an alternative to Kofax?

#3 2009-12-16 14:17:16

Re: Interested in an alternative to Kofax?

#4 2009-12-16 17:13:27

Re: Interested in an alternative to Kofax?

#5 2009-12-16 18:38:12

Re: Interested in an alternative to Kofax?

#6 2009-12-16 19:59:31

Re: Interested in an alternative to Kofax?

#7 2009-12-17 03:10:46

Re: Interested in an alternative to Kofax?

#8 2009-12-17 14:33:59

Re: Interested in an alternative to Kofax?

#9 2009-12-17 23:27:31

Re: Interested in an alternative to Kofax?

#10 2009-12-18 01:17:50

Re: Interested in an alternative to Kofax?

#11 2009-12-18 11:12:18

Re: Interested in an alternative to Kofax?

#12 2009-12-18 19:31:03

Re: Interested in an alternative to Kofax?

#13 2009-12-19 00:12:38

Re: Interested in an alternative to Kofax?

#14 2009-12-19 01:25:55

Re: Interested in an alternative to Kofax?

#15 2009-12-20 21:34:33

Re: Interested in an alternative to Kofax?

#16 2009-12-22 04:58:58

Re: Interested in an alternative to Kofax?

#17 2009-12-23 15:55:29

Re: Interested in an alternative to Kofax?

#18 2009-12-29 15:18:17

Re: Interested in an alternative to Kofax?

#19 2010-01-01 16:35:47

Re: Interested in an alternative to Kofax?

#20 2010-01-06 10:25:01

Re: Interested in an alternative to Kofax?

#21 2010-02-14 20:03:10

Re: Interested in an alternative to Kofax?

#22 2011-01-15 16:18:52

Re: Interested in an alternative to Kofax?

Board footer