Latent semantic analysis - Wikipedia, the free encyclopedia

06 Mar 15

Ryan Muller

31 Jul 14

Latent semantic indexing (LSI) is an indexing and retrieval method that uses a mathematical technique called singular value decomposition (SVD) to identify patterns in the relationships between the terms and concepts contained in an unstructured collection of text.

search seo

25 Jul 14

ralawami

Indexing and retrieval method
relationships between the terms nad concepts contained in an unsrtuctured collection of text

im-tutorial

27 Mar 14

Daniel Mulder

26 Feb 14

carlos puentes

"Latent semantic indexing (LSI) is an indexing and retrieval method that uses a mathematical technique called singular value decomposition (SVD) to identify patterns in the relationships between the terms and concepts contained in an unstructured collection of text. LSI is based on the principle that words that are used in the same contexts tend to have similar meanings. A key feature of LSI is its ability to extract the conceptual content of a body of text by establishing associations between those terms that occur in similar contexts."

28 Jan 14

vmahadev

is an indexing and retrieval method that uses a mathematical technique called singular value decomposition (SVD) to identify patterns in the relationships between the terms and concepts contained in an unstructured collection of text. LSI is based on the principle that words that are used in the same contexts tend to have similar meanings. A key feature of LSI is its ability to extract the conceptual content of a body of text by establishing associations between those terms that occur in similar contexts.
Latent Semantic Indexing because of its ability to correlate semantically related terms that are latent in a collection of text, it was first applied to text at Bell Laboratories in the late 1980s. The method, also called latent semantic analysis (LSA), uncovers the underlying latent semantic structure in the usage of words in a body of text and how it can be used to extract the meaning of the text in response to user queries, commonly referred to as concept searches.

12 Jun 13

Bruno Melo

semantic analysis indexing nlp ai latent text ir

11 Mar 13

Carl Potts

semantic indexing analysis

14 Jan 13

Jerry Milo Johnson

semantic analysis lsi content

11 Oct 12

Marco Antonio Almeida

08 Apr 12

deeptigp

Early challenges to LSI focused on scalability and performance. LSI requires relatively high computational performance and memory in comparison to other information retrieval techniques

29 Feb 12

astray0924

ir

Latent Semantic Indexing (LSI) is an indexing and retrieval method that uses a mathematical technique called Singular value decomposition (SVD) to identify patterns in the relationships between the terms and concepts contained in an unstructured collection of text. LSI is based on the principle that words that are used in the same contexts tend to have similar meanings. A key feature of LSI is its ability to extract the conceptual content of a body of text by establishing associations between those terms that occur in similar contexts.^[1]

30 Dec 11

Marc-Alexandre Gagnon

semantic analysis indexing nlp ai latent text lsa

07 Dec 11

William Jones

of dimensions to use for performing the SVD.

16 Sep 11

marco_antonio_almeida_silva

analysis nlp language algorithms ai semanticweb algorithm semantic lsi lsa

29 Jul 11

Laki Sideris

semantic LS

Because it uses a strictly mathematical approach, LSI is inherently independent of language. This enables LSI to elicit the semantic content of information written in any language without requiring the use of auxiliary structures, such as dictionaries and thesauri. LSI can also perform cross-
005 – First vertical-specific application – publishing – EDB (EBSCO, Content Analyst Company)
- Information discovery^[18] (eDiscovery, Government/Intelligence community, Publishing)
- Automated document classification (eDiscovery, Government/Intelligence community, Publishing)^[19]
- Text summarization^[20] (eDiscovery, Publishing)

1 more annotation...

11 Jul 11

alarbu

semantic analysis latent text indexing ai DataMining

Latent Semantic Indexing (LSI) is an indexing and retrieval method that uses a mathematical technique called Singular value decomposition (SVD) to identify patterns in the relationships between the terms and concepts contained in an unstructured collection of text. LSI is based on the principle that words that are used in the same contexts tend to have similar meanings. A key feature of LSI is its ability to extract the conceptual content of a body of text by establishing associations between those terms that occur in similar contexts.
Benefits of LSI

LSI overcomes two of the most problematic constraints of Boolean keyword queries: multiple words that have similar meanings (synonymy) and words that have more than one meaning (polysemy). Synonymy and polysemy are often the cause of mismatches in the vocabulary used by the authors of documents and the users of information retrieval systems.^[3] As a result, Boolean keyword queries often return irrelevant results and miss information that is relevant.

15 Apr 11

uycmkwp

Latent Semantic Indexing (LSI) is an indexing and retrieval method that uses a mathematical technique called Singular Value Decomposition (SVD) to identify patterns in the relationships between the terms and concepts contained in an unstructured collection of text. LSI is based on the principle that words that are used in the same contexts tend to have similar meanings. A key feature of LSI is its ability to extract the conceptual content of a body of text by establishing associations between those terms that occur in similar contexts.^[1]
Term Document Matrix

LSI begins by constructing a term-document matrix, $A$ , to identify the occurrences of the $m$ unique terms within a collection of $n$ documents. In a term-document matrix, each term is represented by a row, and each document is represented by a column, with each matrix cell, $a i j$ , initially representing the number of times the associated term appears in the indicated document, $tf ij$ . This matrix is usually very large and very sparse.
Once a term-document matrix is constructed, local and global weighting functions can be applied to it to condition the data.
Some common local weighting functions ^[13] are defined in the following table.

Binary $l i j = 1$ if the term exists in the document, or else $0$
TermFrequency $l i j = tf i j$ , the number of occurrences of term $i$ in document $j$
Log $l i j = log(tf i j + 1)$
Augnorm $l_{ij} = \frac{\Big(\frac{\mathrm{tf}_{ij}}{\max_i(\mathrm{tf}_{ij})}\Big) + 1}{2}$

Some common global weighting functions are defined in the following table.

Binary $g i = 1$
Normal $g_i = \frac{1}{\sqrt{\sum_j \mathrm{tf}_{ij}^2}}$
GfIdf $g i = gf i / df i$ , where $gf i$ is the total number of times term $i$ occurs in the whole collection, and $df i$ is the number of documents in which term $i$ occurs.
Idf $g_i = 1+ \log_2 \frac{n}{\mathrm{df}_i}$
Entropy $g_i = 1 + \sum_j \frac{p_{ij} \log p_{ij}}{\log n}$ , where $p_{ij} = \frac{\mathrm{tf}_{ij}}{\mathrm{gf}_i}$
Empirical studies with LSI report that the Log Entropy weighting functions work well, in practice, with many data sets^[14]. In other words, each entry $a i j$ of $A$ is computed as:

$g_i = 1 + \sum_j \frac{p_{ij} \log p_{ij}}{\log n}$

$a_{ij} = g_i \ \log (\mathrm{tf}_{ij} + 1)$

[edit]

Binary		$l i j = 1$ if the term exists in the document, or else $0$
TermFrequency		$l i j = tf i j$ , the number of occurrences of term $i$ in document $j$
Log		$l i j = log(tf i j + 1)$
Augnorm		$l_{ij} = \frac{\Big(\frac{\mathrm{tf}_{ij}}{\max_i(\mathrm{tf}_{ij})}\Big) + 1}{2}$

Binary		$g i = 1$
Normal		$g_i = \frac{1}{\sqrt{\sum_j \mathrm{tf}_{ij}^2}}$
GfIdf		$g i = gf i / df i$ , where $gf i$ is the total number of times term $i$ occurs in the whole collection, and $df i$ is the number of documents in which term $i$ occurs.
Idf		$g_i = 1+ \log_2 \frac{n}{\mathrm{df}_i}$
Entropy		$g_i = 1 + \sum_j \frac{p_{ij} \log p_{ij}}{\log n}$ , where $p_{ij} = \frac{\mathrm{tf}_{ij}}{\mathrm{gf}_i}$

3 more annotations...

20 Dec 10

nigifabio

semantic latent analysis nlp indexing vectors nltk text

11 Dec 10

ed lenert

atent Semantic Indexing (LSI) is an indexing and retrieval method that uses a mathematical technique called Singular Value Decomposition (SVD) to identify patterns in the relationships between the terms and concepts contained in an unstructured collection of text.

research

04 Dec 10

kamyar

machine-learning ai

17 Oct 10

Uwe Crenze

semantic lsi nlp analysis indexing conceptsearch lsa

30 Aug 10

Vin M

wcs instantpersonalization

07 Aug 10

davidmerrick

Latent Semantic Indexing (LSI) is an indexing and retrieval method that uses a mathematical technique called Singular Value Decomposition (SVD) to identify patterns in the relationships between the terms and concepts contained in an unstructured collectio

seo ai google algorithm formula data linguistics lsa lsi matrix classification latent

24 Jun 10

Morgan Keys

Latent Semantic Indexing (LSI) is an indexing and retrieval method that uses a mathematical technique called Singular Value Decomposition (SVD) to identify patterns in the relationships between the terms and concepts contained in an unstructured collectio

ai algorithms data language clustering indexing reference wikipedia SVD singular value decomposition

21 Jun 10

scmoral

semantic lsi nlp algorithms ai semanticweb indexing textmining textanalysis ir lsa

15 May 10

urza urza

Latent Semantic Indexing (LSI) is an indexing and retrieval method that uses a mathematical technique called Singular Value Decomposition (SVD) to identify patterns in the relationships between the terms and concepts contained in an unstructured collectio

semantic

14 Jan 10