brent gg's Bookmarks tagged ai →  View Popular

Some examples of the problems faced by natural-language-understanding systems:





• The sentences We gave the monkeys the bananas because they were hungry and We gave the monkeys the bananas because they were over-ripe have the same surface grammatical structure. However, the pronoun they refers to monkeys in one sentence and bananas in the other, and it is impossible to tell which without a knowledge of the properties of monkeys and bananas.


• A string of words may be interpreted in different ways. For example, the string Time flies like an arrow may be interpreted in a variety of ways:
  
  
  
  • The common simile: time moves quickly just like an arrow does;
  
  
  • measure the speed of flying insects like you would measure that of an arrow (thus interpreted as an imperative) - i.e. (You should) time flies as you would (time) an arrow.;
  
  
  • measure the speed of flying insects like an arrow would - i.e. Time flies in the same way that an arrow would (time them).;
  
  
  • measure the speed of flying insects that are like arrows - i.e. Time those flies that are like arrows;
  
  
  • all of a type of flying insect, "time-flies," collectively enjoys a single arrow (compare Fruit flies like a banana);
  
  
  • each of a type of flying insect, "time-flies," individually enjoys a different arrow (similar comparison applies);
  
  
  • A concrete object, for example the magazine, Time, travels through the air in an arrow-like manner.
  
  
  
  





English is particularly challenging in this regard because it has little inflectional morphology to distinguish between parts of speech.





• English and several other languages don't specify which word an adjective applies to. For example, in the string "pretty little girls' school".
  
  
  
  • Does the school look little?
  
  
  • Do the girls look little?
  
  
  • Do the girls look pretty?
  
  
  • Does the school look pretty?
  
  
  
  






• We will often resolve ambiguities in language by the way we place stress on words. The sentence "I never said she stole my money" demonstrates the importance stress can play in a sentence, and thus the inherent difficulty a natural language processor can have in parsing it. Depending on which word the speaker places the stress, this sentence could have several distinct meanings:
  
  
  
  • "I never said she stole my money" - Someone else said it, but I didn't.
  
  
  • "I never said she stole my money" - I simply didn't ever say it.
  
  
  • "I never said she stole my money" - I might have implied it in some way, but I never explicitly said it.
  
  
  • "I never said she stole my money" - I said someone took it, I didn't say it was her.
  
  
  • "I never said she stole my money" - I just said she probably borrowed it.
  
  
  • "I never said she stole my money" - I said she stole someone else's money.
  
  
  • "I never said she stole my money" - I accused her of stealing my heart, but not my money.

Learning algorithms

Training a neural network model essentially means selecting one model from the set of allowed models (or, in a Bayesian framework, determining a distribution over the set of allowed models) that minimises the cost criterion. There are numerous algorithms available for training neural network models; most of them can be viewed as a straightforward application of optimization theory and statistical estimation.

Most of the algorithms used in training artificial neural networks are employing some form of gradient descent. This is done by simply taking the derivative of the cost function with respect to the network parameters and then changing those parameters in a gradient-related direction.

Evolutionary methods, simulated annealing, and expectation-maximization and non-parametric methods are among other commonly used methods for training neural networks. See also machine learning.