I'm using OCR to recognize (German) text in an image. It works well but not perfectly. Sometimes a word gets messed-up. Therefore, I want to implement some sort of validation. Of course, I can just use a word list and find words that are similar to the messed-up word, but is there a way to check if the sentence is plausible with these words?
After all, my smartphone can give me good suggestions on how to complete a sentence.
You need to look for Natural Language Processing (NLP) solutions. With them, you can validate syntactically the lexical (either the whole text, which may be better as some of them may take on consideration the context, or phrase by phrase).
I am not an expert in the area, but this article can help you to choose a tool to start trying.
Also, please notice: your keyboard on your cellphone is developed and maintained by specialized teams, either on Apple, Google or any other company that you use their app. So, please, don't underestimate this task: there are dozens of research areas on this, that includes either software engineers and linguistics specialists to achieve proper results.
Edit: well, two days later, I've just came to this link: https://medium.com/quick-code/12-best-natural-language-processing-courses-2019-updated-2a6c28aebd48
I need to create a DAWG graphic for my scrabble IA.
After multi search I found two or three sites that explain how to create a DAWG:
https://progaide.com/question/12331755-algorithme-de-cr-ation-de-dawg-facile
How to create a DAWG?
https://codes-sources.commentcamarche.net/faq/10903-compression-d-un-dictionnaire-sous-forme-de-dawg
However, I don't understand it really well.
I need to create this graph with a dictionary in .txt (approximately 400.000 word like a french dictionary) to optimize a search of the different words that existe in the French Language.
Now I have a simple search in my .txt but it is really slow and i think that the real good method to implement this, is a DAWG.
My IA can put the first word with the letter of player1 but for the other turn of play i need to analyze a word with 8 letter and not 7, and i think the best option before advanced is optimize my research. For me, the best solution is a DAWG but if you have any other solution, I am open minded.
Thanks for reading me, I hope my english is understandable.
PS: if one need the .txt, I can give it to you. No problem (is a real full french dictionary for scrabble)
I have an idea to build program than can interact with the user voice in Arabic language, since one year ,I started with sphinx-4 but I need to make arabic acoustic model , grammar , dictionary. .
but I can't find the rood I want you to tell me in detailed description how to create those things?
the needed iIDE or program
please help me....
Ok, let me start at the very beginning, because I think you are not aware of the dimensions of your project, and you are mixing up things (ASR and TTS). First, I would like to explain what the different things are that you were talking about:
Acoustic Model: Every speech recognition system requires an acoustic model. Language, in particular words, are made up of phonemes. Phonemes describe how something sounds. To give you an example, the letter a is not always pronounced the same way, as you can see from the two words below:
to bark <=> to take
Now your ASR system needs to detect these phonemes. To do this, it performs a spectral analysis of many short frames of the audio signal and computes features, like MFCCs. What to do with these features? It puts them into a classifier (I could write a new chapter about the classifier here, but this will be too much information). Your classifier has to learn how to actually perform classification. What it does in simple words is it maps a set of features to a phoneme.
Dictionary: In your dictionary, you define every word that can be recognized by your ASR system. It tells the ASR the phoneme composition of a word. A short example for this is:
hello H EH L OW
world W ER L D
With this small dictionary, your system would be able to recognize the words hello and world.
Language Model (or Grammar): The language model holds information about the assembly of words for a given language. What does this mean? Think of the virtual keyboard of your smartphone. When you type in the words 'Will you marry', your keyboard might guess the next word to be 'me'. That is no magic. The model was learned from huge amounts of text files. Your LM does the same. It adds the knowledge about meaningful word compositions (what everybody calls a sentence) into the ASR system to further improve detection.
Now back to your problem: You need transcribed audio data for the following reasons:
You want to train your acoustic model if you have none.
You want to create a large enough dictionary.
You want to generate a language model from the text.
Long story short: You are wrong if you think you could solve all these tasks on your own. Only a reliable transcription is already a large amount of work. You should clearly overthink your idea.
Im looking for a speech recognition software for java that acts more like the android version, in that, instead of having .gram files and stuff, it just returns a string of what was said, and I can act on it. Ive tried using sphinx-4, but using .gram files makes my program a lot harder to do.
The point of a grammar file is to improve the accuracy of what you're getting back. Instead of trying to come up with random strings of english words, you tell it to expect specific input.
That said, sphinx-4 can do ordinary large-dictionary ASR as well. Read the N-Gram part of this tutorial and look at the Transcriber sample that comes with the sphinx source code.
In addition, you can train your own trigram model that will enhance the results you get. (E.g., place more probability on the word "weather" being detected.) This is certainly what Siri does. Apple/Google have a huge corpus of pieces of audio that people speak into their phones, part of which is human transcribed, from which they train both acoustic and linguistic models (so their engines detect things people typically say instead of nonsense).
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I realize that to become a better programmer, you need to program!
So obviously the more practice, the better you become.
My problem is this. I am currently in university, and I find my course load is a bit daunting, and I don't have a lot of free time. I don't think I could really take on a big project, particularly I don't think I would have to motivation to see it through, it would be easier just for me to keep putting it off in favour of work that is due is school.
But I still want to practice.
So I am looking for any resources which have programming challenges which can be completed in a fairly small amount of time. Ideally something i could get done in under 10 hours of work (so just over an hour of work each day), if not smaller.
I have heard of Google Code Jam, but I am not sure the length of the programs it specifies, nor the skill level.
Does anyone have suggestions? Even perhaps a compendium of tutorials for different functions might be useful. For example, a tutorial on file IO would be worthwhile (if I didn't already know it), even though it can be a fairly small topic.
You should look into code katas, they do exactly what you are talking about. Short exercises that are designed to perfect your coding/thinking abilities.
Other references:
http://kata.coderdojo.com/wiki/Overview_of_Learning_Resources
Project Euler has some math/number related problems that are very interesting and ranged from easy to very challenging. You can pick your language of choice and submit only the solution (a large integer number). After you submitted the correct solution, you have access to a forum/comment page where others posted their comments and solutions.
From experience I recommend finding a task that you do repetitively and turning it into a program. I also recommend, seriously, re-invent the wheel in order to get practice with programming. Don't let people tell you to not do something just because it exists already. If you don't know how it works, try to write it yourself.
I don't exactly know what programming level you are on, but don't try to do anything too crazy off the bat, that is just a demotivator (such as trying to write a game for the PS3).
If you already can navigate your way around with IO, then you should try to really learn how to use Collections effectively. I think one of the best practice assignments I have ever done was rewriting the Java TreeMap Class. It was a huge challenge and I learned a lot by doing it.
Here are some suggestions for practice assignments:
Take a text file that has a fair amount of information in it, grab anything, you can get something from here if you'd like: http://www.gutenberg.org/ and make a program that will do the following:
Read in the file
Create a collection of words and their occurrences
Create a collection of anagrams
Create a collection of words and the positions in which they occur (line#, word position)
Develop statistics on the words in the file - meaning - treating each word as an individual - which words occur before it and after it.
Remove all of the white space from the file
Write all of the above data to their own files
One of my favorite things to do is mess with web data, go to a polling website, find a page that has poll data in a tabular form and do the following:
Download the data
Parse through the data and turn the tabular data into a CSV file
Open it in excel without error
Or just look for any site and extract data from it, just make sure the site is robot friendly http://www.robotstxt.org/, you don't want any one site to feel like it is under attack. Most of the time though this isn't normally a problem because if you read the site's terms of use it clearly states you are allowed to download 1 copy of whatever it is you are viewing so long as you don't intend to sell it. Of course this changes for every site.
Go to a website and get all of the links off of the page programmatically.
Here is a fun one, the Susan Program (I don't remember why it is named Susan) which I initially wrote using a C program and two Bourne shell scripts in a Unix environment. The idea in this program is to fork 4 child processes and give them each a task like so:
Child 1: Reads in a file, creates a dictionary of each word and its position in the file, this is outputted to a file.
Child 2: Takes Child 1's output and reconstructs the document, this is outputted to a file.
Child 3: Takes Child 2's output and does what child 1 did again
Child 4: Takes Child 3's output and does what child 2 did again
The goal here is to have an exact replica of the original file once Child 4 outputs it. This is challenging and somewhat pointless, but the point of this exercise is to get the practice.
In your case, don't feel that you need to use different threads for this, you can just use a single program with two different functions and just call them in order.
Again, not sure if you are at this level yet, but try to replace any "for" or "foreach" loop you have in your program with recursion, just as practice. Recursion is a pain in the butt, but it is valuable to know and understand.
These are some suggestions which I think will really help you sharpen your skills.
Enjoy
I like SPOJ and Project Euler to take quick programming challenges and exercises.
Code Jam is a good programming contest, although, as you mentioned, most of the problems there aren't for beginners.
There's a good selection of problems from past topcoder algorithm competitions. (They are held ~2 times a month for almost 10 years already, so there're quite a lot.)
Difficulty range from very simple (but still interesting) problems in the 2nd division to very hard.
Additionally, there're editorials with solutions and live environment where you can submit and test your code. You can also learn from submissions by other people.
Check the problem listing.
Another advantage of topcoder is the regular online contests they hold. I find that competing against other people in realtime is a great boost for motivation.
There're many more problem archives, like SPOJ, UVA and Timus, although they rarely provide solutions or even hints.
http://codegolf.stackexchange.com might have some programming challenges to your liking. A lot of the answers on that site are golfed (they implement the program in the least number of characters) but there are definitely some interesting examples to learn from.
Try enrolling on any IT course on the following websites:
Coursera
Edx
Udacity
These websites offer free educational IT programs from prestigious schools wherein there are lot of challenging exercises to sharpen your programming skills. I've learned to program percolation, pattern recognition, bouncing ball and so many more interesting things because of this. You will upload your program upon completion of the exercises and you will be graded accordingly (basically your progam will be checked).
At the end of each course, you will even receive a certificate of completion. Cool Right?
It depends of the language, but in the past http://rubyquiz.com and http://pythonchallenge.com did great for me, also you can join to an open source initiative because usually helps to give you better code review chances.
I've always thought that practicing with sample interview questions was a great way to sharpen one's skills and get exposed to types of problems that you normally wouldn't solve. Plus, if you're going to be looking for a job it helps you even more.
Here's a pretty simple one that I did for fun the other day:
Write a routine to print the numbers 1
to 100 and back to 1 again without
using any loops.
Glassdoor.com has a lot of good interview question submitted by people who actually got them in an interview.
Since you are in University and looking to improve your coding skills the hard-copy book Cracking the Coding Interview might be a good fit for you. It's got great general programming questions and tidbits about interviewing with some of the best companies in tech. Not only are there great questions, but there are decent problem breakdowns as well.
[Disclosure: I own the book but otherwise have no association to it.]
If you like programming and want to improve your programmer skills, you must try cocode.co. It's a social young site, similar to StackOverflow but based on posting and solving programming challenges, instead of asking and answering questions. From very easy challenges to very hard ones.
You can try to solve ACM problems. There are thousands of problems there and you can find the difficulty level so you can choose which problems to do first. The offcial site for this is:
http://uva.onlinejudge.org/. You can learn more there.
regards
arefin
It may seem a little obvious, but I've noticed a real boost in my regular-expressions skills lately just from answering regex questions on Stack Overflow. Teaching forces you to break down problems into easily explainable pieces, and will also guide your research on those occasions where you know most, but not quite all, of a solution.
I suggest finding a topic you're already somewhat proficient in, since this type of thing isn't so good as a beginners' tutorial. Search SO for questions tagged with that topic and try to figure out the answers. Don't just code them in your head; go ahead and write them out, test them, and explain them. If you're not sure your answer is correct, just write it without posting it.