Monday, May 14, 2012

opinion mining -sentiment mining

Opinion mining is a type of natural language processing for tracking the mood of the public about a particular product. Opinion mining, which is also called sentiment analysis, involves building a system to collect and examine opinions about the product made in blog posts, comments, reviews or tweets.  Automated opinion mining often uses machine learning, a component of artificial intelligence (AI).
Opinion mining can be useful in several ways.  If you are in marketing, for example, it can help you judge the success of an ad campaign or new product launch, determine which versions of a product or service are popular and even identify which demographics like or dislike particular features. For example, a review might be broadly positive about a digital camera, but be specifically negative about how heavy it is. Being able to identify this kind of information in a systematic way gives the vendor a much clearer picture of public opinion than surveys or focus groups, because the data is created by the customer.  
An opinion mining system is often built using software that is capable of extracting knowledge from examples in a database and incorporating new data to improve performance over time.  The process can be as simple as learning a list of positive and negative words, or as complicated as conducting deep parsing of the data in order to understand the grammar and sentence structure used.  
There are several challenges in opinion mining. The first is that a word that is considered to be positive in one situation may be considered negative in another situation. Take the word "long" for instance. If a customer said a laptop's battery life was long, that would be a positive opinion.  If the customer said that the laptop's start-up time was long, however, that would be is a negative opinion. These differences mean that an opinion system trained to gather opinions on one type of product or product feature may not perform very well on another. 
A second challenge is that people don't always express opinions the same way. Most traditional text processing relies on the fact that small differences between two pieces of text don't change the meaning very much.  In opinion mining, however, "the movie was great" is very different from "the movie was not great". 
Finally, people can be contradictory in their statements. Most reviews will have both positive and negative comments, which is somewhat manageable by analyzing sentences one at a time. However, the more informal the medium (twitter or blogs for example), the more likely people are to combine different opinions in the same sentence. For example: "the movie bombed even though the lead actor rocked it" is easy for a human to understand, but more difficult for a computer to parse. Sometimes even other people have difficulty understanding what someone thought based on a short piece of text because it lacks context.  For example, "That movie was as good as his last one" is entirely dependent on what the person expressing the opinion thought of the previous film.

Job Opportunities in Data Mining


Software Developer in Data Mining Technologies


The Oracle Data Mining group is responsible for embedding data mining technology within the Oracle database. We currently have a large breadth of mining functions and algorithms embedded in the Oracle database:
- classification (decision tree, support vector machine, naive bayes, and logistic regression)
- regression (support vector machine and multivariate linear regression)
- clustering (enhanced k-means and o-cluster, an Oracle developed algorithm)
- feature extraction (non-negative matrix factorization)
- associations (apriori)
- attribute importance (mdl)
In addition, we have enhanced Oracle's native, built-in SQL language to support functions which score data mining models (e.g., produce the probability of churn).

Expected background:

strong background in machine learning (Ph.D. preferable)

strong background in mathematics

strong knowledge of C

knowledge of SQL is a plus

Is Matlab best language for Data mining?


While starting a new project a few days ago, I had to answer the recurrent question: What language do I choose? In research, we have the opportunity of choosing any language, free or not. This is usually not the case in industry where the language can be fixed for many reasons (price, customer choice, boss choice, same as existing system, etc.).

I basically had to choose between Java and Matlab (C++ was soon deleted from my list since I don’t like to spend time on pointers and manually free up the memory, but this is very personal). Of course a lot of others are available, but I feel more confident with these two. As most of my work was done with Matlab, I decided to start with Java. Contradictory? Not at all, I just wanted to know how easy it was to use Java for raw data mining tasks (i.e. without using JDM framework or such).

When doing data mining, a large part of the work is to manipulate data. Indeed, the part of coding the algorithm can be quite short since Matlab has a lot of toolboxes for data mining. And when manipulating data, Matlab is definitely better. It is normal since it is done to work with matrices (MATrix LABoratory). Thus, deleting a row, a column, transposing a matrix, calculating the determinant… all these can be done in one line of code. To my knowledge, this is not the case with Java, but if you know some way, feel free to comment.

For more information about using Matlab for data mining, the best place is
http://matlabdatamining.blogspot.in/.