PYTHON PACKAGES FOR DATA MINING

The intelligent key thing is when you use  the same hammer to solve what ever problem you came across. Like the same way when we indented to solve a data mining problem  we will face so many issues but we can solve them by using python in a intelligent way.

Before stepping directly to Python packages, let me clear up any doubts you may have about why you should be using Python.

WHY PYTHON ?

We all know that python is powerful programming language, but what does that mean, exactly? What makes python  a powerful programming language?

PYTHON IS EASY

Universally, Python has gained a reputation because of it’s easy to learn. The syntax of Python programming language is designed to be easily readable. Python has significant popularity in  scientific computing. The people working in this field are scientists first, and programmers second.

PYTHON IS EFFICIENT

Nowadays we working on bulk amount of data, popularly known as big data.  The more data you have to process, the more important it becomes to manage the memory you use. Here Python will work very efficiently.

PYTHON IS FAST

We all know Python is an interpreted language, we may think that it is slow, but some amazing work has been done over the past years to improve Python’s performance. My point is that if you want to do high-performance computing, Python is a viable best option today.

Hope I cleared your doubt about “Why Python?”, so let me jump to Python Packages for data mining.

NUMPY

 About:

NumPy is the fundamental package for scientific computing with Python. NumPy is an extension to the Python programming language, adding support for large, multi-dimensional arrays and matrices, along with a large library of high-level mathematical functions to operate on these arrays. The ancestor of NumPy, Numeric, was originally created by Jim Hugunin with contributions from several other developers. In 2005, Travis Oliphant created NumPy by incorporating features of the competing Numarray into Numeric, with extensive modifications.

Original author(s)	Travis Oliphant
Developer(s)	Community project
Initial release	As Numeric, 1995; as NumPy, 2006
Stable release	1.9.0 / 7 September 2014; 36 days ago
Written in	Python, C
Operating system	Cross-platform
Type	Technical computing
License	BSD-new license
Website	www.numpy.org

Installing numpy:

If Python is not installed in your computer please install it first.

Installing numpy in linux

Open your terminal and copy these commands:

sudo apt-get update
sudo apt-get install python-numpy

Sample numpy code for using reshape function

[code language=”css”]from numpy import *
a = arange(12)
a = a.reshape(3,2,2)
print a [/code]

Script output

[[[ 0 1]
[ 2 3]]

[[ 4 5]
[ 6 7]]

[[ 8 9]
[10 11]]]

SCIPY

About:

SciPy (pronounced “Sigh Pie”) is open-source software for mathematics, science, and engineering. The SciPy library depends on NumPy, which provides convenient and fast N-dimensional array manipulation. The SciPy library is built to work with NumPy arrays, and provides many user-friendly and efficient numerical routines such as routines for numerical integration and optimization. Together, they run on all popular operating systems, are quick to install, and are free of charge. NumPy and SciPy are easy to use, but powerful enough to be depended upon by some of the world’s leading scientists and engineers. If you need to manipulate numbers on a computer and display or publish the results, Scipy is the tool for the job.

Original author(s)	Travis Oliphant, Pearu Peterson, Eric Jones
Developer(s)	Community library project
Stable release	0.14.0 / 3 May 2014; 5 months ago
Written in	Python, Fortran, C, C++[1]
Operating system	Cross-platform (list)
Type	Technical computing
License	BSD-new license
Website	www.scipy.org

 Installing SciPy in linux

Open your terminal and copy these commands:

sudo apt-get update
sudo apt-get install python-scipy

Sample SciPy code

[code language=”css”] from scipy import special, optimize
f = lambda x: -special.jv(3, x)
sol = optimize.minimize(f, 1.0)
x = linspace(0, 10, 5000)
plot(x, special.jv(3, x), ‘-‘, sol.x, -sol.fun, ‘o’)
savefig(‘plot.png’, dpi=96)[/code]

 Script output

PANDAS

About:

Pandas is a Python package providing fast, flexible, and expressive data structures designed to make working with “relational” or “labeled” data both easy and intuitive. It aims to be the fundamental high-level building block for doing practical, real world data analysis in Python. Additionally, it has the broader goal of becoming the most powerful and flexible open source data analysis / manipulation tool available in any language. It is already well on its way toward this goal.

Pandas is well suited for many different kinds of data:

• Tabular data with heterogeneously-typed columns, as in an SQL table or Excel spreadsheet.
• Ordered and unordered (not necessarily fixed-frequency) time series data.
• Arbitrary matrix data (homogeneously typed or heterogeneous) with row and column labels.
• Any other form of observational / statistical data sets. The data actually need not be labeled at all to be placed into a pandas data structure.

Installing Pandas in Linux

Open your terminal and copy these commands:

sudo apt-get update
sudo apt-get install python-pandas

Sample Pandas code about Pandas Series

[code language=”css”]import pandas as pd

values = np.array([2.0, 1.0, 5.0, 0.97, 3.0, 10.0, 0.0599, 8.0])
ser = pd.Series(values)
print ser[/code]

Script output
0 2.0000
1 1.0000
2 5.0000
3 0.9700
4 3.0000
5 10.0000
6 0.0599
7 8.0000

MATPLOTLIB

About:
matplotlib is a plotting library for the Python programming language and its NumPy numerical mathematics extension. It provides an object-oriented API for embedding plots into applications using general-purpose GUI toolkits like wxPython, Qt, or GTK+. There is also a procedural “pylab” interface based on a state machine (like OpenGL), designed to closely resemble that of MATLAB. SciPy makes use of matplotlib.

Original author(s)	John Hunter
Developer(s)	Michael Droettboom, et al.
Stable release	1.4.2 (26 October 2014; 3 days ago) [±]
Written in	Python
Operating system	Cross-platform
Type	Plotting
License	matplotlib license
Website	matplotlib.org

Installing Matplotlib in linux

Open your terminal and copy these commands:

sudo apt-get update
sudo apt-get install python-matplotlib

Sample Matplotlib code to Create Histograms

[code language=”css”]import numpy as np
import matplotlib.mlab as mlab
import matplotlib.pyplot as plt

# example data
mu = 100 # mean of distribution
sigma = 15 # standard deviation of distribution
x = mu + sigma * np.random.randn(10000)

num_bins = 50
# the histogram of the data
n, bins, patches = plt.hist(x, num_bins, normed=1, facecolor=’green’, alpha=0.5)
# add a ‘best fit’ line
y = mlab.normpdf(bins, mu, sigma)
plt.plot(bins, y, ‘r–‘)
plt.xlabel(‘Smarts’)
plt.ylabel(‘Probability’)
plt.title(r’Histogram of IQ: $\mu=100$, $\sigma=15$’)

# Tweak spacing to prevent clipping of ylabel
plt.subplots_adjust(left=0.15)
plt.show()[/code]

Script output

IPYTHON

IPython is a command shell for interactive computing in multiple programming languages, originally developed for the Python programming language, that offers enhanced introspection, rich media, additional shell syntax, tab completion, and rich history. IPython currently provides the following features:

• Powerful interactive shells (terminal and Qt-based).
• A browser-based notebook with support for code, text, mathematical expressions, inline plots and other rich media.
• Support for interactive data visualization and use of GUI toolkits.
• Flexible, embeddable interpreters to load into one’s own projects.
• Easy to use, high performance tools for parallel computing.

Original author(s)	Fernando Perez and others
Stable release	2.3 / 1 October 2014; 27 days ago
Written in	Python, JavaScript, CSS,HTML
Operating system	Cross-platform
Type	Shell
License	BSD
Website	www.ipython.org

Installing IPython in linux

Open your terminal and copy these commands:

sudo apt-get update
sudo pip install ipython

Sample IPython code

This piece of code is to plot demonstrating the integral as the area under a curve

[code language=”css”]import numpy as np
import matplotlib.pyplot as plt
from matplotlib.patches import Polygon
def func(x):
return (x – 3) * (x – 5) * (x – 7) + 85

a, b = 2, 9 # integral limits
x = np.linspace(0, 10)
y = func(x)

fig, ax = plt.subplots()
plt.plot(x, y, ‘r’, linewidth=2)
plt.ylim(ymin=0)

# Make the shaded region
ix = np.linspace(a, b)
iy = func(ix)
verts = [(a, 0)] + list(zip(ix, iy)) + [(b, 0)]
poly = Polygon(verts, facecolor=’0.9′, edgecolor=’0.5′)
ax.add_patch(poly)

plt.text(0.5 * (a + b), 30, r"$\int_a^b f(x)\mathrm{d}x$",
horizontalalignment=’center’, fontsize=20)

plt.figtext(0.9, 0.05, ‘$x$’)
plt.figtext(0.1, 0.9, ‘$y$’)

ax.spines[‘right’].set_visible(False)
ax.spines[‘top’].set_visible(False)
ax.xaxis.set_ticks_position(‘bottom’)

ax.set_xticks((a, b))
ax.set_xticklabels((‘$a$’, ‘$b$’))
ax.set_yticks([])

plt.show()[/code]

Script output

SCIKIT-LEARN

The scikit-learn project started as scikits.learn, a Google Summer of Code project by David Cournapeau. Its name stems from the notion that it is a “SciKit” (SciPy Toolkit), a separately-developed and distributed third-party extension to SciPy. The original codebase was later extensively rewritten by other developers. Of the various scikits, scikit-learn as well as scikit-image were described as “well-maintained and popular” in November 2012.

Original author(s)	David Cournapeau
Initial release	June 2007; 7 years ago[1]
Stable release	0.15.1 / August 1, 2014; 2 months ago[2]
Written in	Python, Cython, C andC++
Operating system	Linux, Mac OS X,Microsoft Windows
Type	Library for machine learning
License	BSD License
Website	scikit-learn.org

Installing Scikit-learn in linux

Open your terminal and copy these commands

sudo apt-get update
sudo apt-get install python-sklearn

Sample Scikit-learn code

[code language=”css”]import matplotlib.pyplot as plt
import numpy as np
from sklearn import datasets, linear_model

# Load the diabetes dataset
diabetes = datasets.load_diabetes()

# Use only one feature
diabetes_X = diabetes.data[:, np.newaxis]
diabetes_X_temp = diabetes_X[:, :, 2]

# Split the data into training/testing sets
diabetes_X_train = diabetes_X_temp[:-20]
diabetes_X_test = diabetes_X_temp[-20:]

# Split the targets into training/testing sets
diabetes_y_train = diabetes.target[:-20]
diabetes_y_test = diabetes.target[-20:]

# Create linear regression object
regr = linear_model.LinearRegression()

# Train the model using the training sets
regr.fit(diabetes_X_train, diabetes_y_train)

# The coefficients
print(‘Coefficients: \n’, regr.coef_)
# The mean square error
print("Residual sum of squares: %.2f"
% np.mean((regr.predict(diabetes_X_test) – diabetes_y_test) ** 2))
# Explained variance score: 1 is perfect prediction
print(‘Variance score: %.2f’ % regr.score(diabetes_X_test, diabetes_y_test))

# Plot outputs
plt.scatter(diabetes_X_test, diabetes_y_test, color=’black’)
plt.plot(diabetes_X_test, regr.predict(diabetes_X_test), color=’blue’,
linewidth=3)

plt.xticks(())
plt.yticks(())

plt.show() [/code]

Script output

Coefficients:
[ 938.23786125]
Residual sum of squares: 2548.07
Variance score: 0.47

I have explained the packages which we are going to use in coming posts to solve some interesting problems.

Please leave your comment if you have any other Python data mining packages to add to this list.

Originally published here.

Data Mining - Fruitful and Fun

Orange Tutorial

http://www.orange.biolab.si/tutorial/rst/index.html

DATA MINING 2014

Topics for Core Data Mining 

- Parallel and distributed data mining algorithms
- Data streams mining
- Graph mining
- Spatial data mining
- Text video, multimedia data mining
- Web mining
- Pre-processing techniques
- Visualization
- Security and information hiding in data mining

Theoretical and Application-oriented Topics

• Case-Based Reasoning and Similarity-Based Reasoning
• Clustering
• Classification & Prediction
• Statistical Learning
• Association Rules
• Telecommunication
• Design of Experiment
• Strategy of Experimentation
• Capability Indices
• Deviation and Novelty Detection
• Control Charts
• Conceptional Learning
• Goodness Measures and Evaluation (e.g. false discovery rates)
• Inductive Learning Including Decision Tree and Rule Induction Learning
• Organisational Learning and Evolutional Learning
• Sampling Methods
• Similarity Measures and Learning of Similarity
• Statistical Learning and Neural Net Based Learning
• Visualization and Data Mining
• Deviation and Novelty Detection
• Feature Grouping, Discretization, Selection and Transformation
• Feature Learning
• Frequent Pattern Mining
• Learning and Adaptive Control
• Learning/Adaption of Recognition and Perception
• Learning for Handwriting Recognition
• Learning in Image Pre-Processing and Segmentation
• Mining Financial or Stockmarket Data
• Mining Motion from Sequence
• Subspace Methods
• Support Vector Machines
• Time Series and Sequential Pattern Mining
• Desirabilities
• Graph Mining
• Agent Data Mining
• Applications in Software Testing
• Knowledge Management
• Mining Social Media
• Online Targeting & Controlling
• Behavioral Targeting
• Meteorological Data Mining
• Data Mining in Logistics
• Data Mining in Energy Industry
• Business Intelligence and Data Mining
• Legal Informatics and Data Mining
• Data Mining for Logistic and Supply Chain Management

Applications of Data Mining

• Marketing
• Medicine
• E-Commerce (Mining Logfiles)
• Biotechnology
• Quality Management
• Multimedia Data (Image, Video, Text, Signals)
• Web-Mining
• Intrusion Detection in Networks
• Agriculture
• Meterology

Machine Learning and Data Mining

International Conference on Machine Learning and Data Mining MLDM

The aim of the conference is to bring together researchers from all over the world who deal with machine learning and data mining in order to discuss the recent status of the research and to direct further developments. Basic research papers as well as application papers are welcome. 

All kinds of applications are welcome but special preference will be given to multimedia related applications, biomedical applications, and webmining. MLDM´2013 is the 9th event in a series of MLDM events that have been originally started out as a workshop. Paper submissions should be related but not limited to any of the following topics:

• association rules
• Audio Mining
• case-based reasoning and learning
• classification and interpretation of images, text, video
• conceptional learning and clustering
• Goodness measures and evaluaion (e.g. false discovery rates)
• inductive learning including decision tree and rule induction learning
• knowledge extraction from text, video, signals and images
• mining gene data bases and biological data bases
• mining images, temporal-spatial data, images from remote sensing
• mining structural representations such as log files, text documents and HTML documents
• mining text documents
• organisational learning and evolutional learning
• probabilistic information retrieval
• Selection bias
• Sampling methods
• Selection with small samples
• similarity measures and learning of similarity
• statistical learning and neural net based learning
• video mining
• visualization and data mining
• Applications of Clustering
• Aspects of Data Mining
• Applications in Medicine
• Autoamtic Semantic Annotation of Media Content
• Bayesian Models and Methods
• Case-Based Reasoning and Associative Memory
• Classification and Model Estimation
• Content-Based Image Retrieval
• Decision Trees
• Deviation and Novelty Detection
• Feature Grouping, Discretization, Selection and Transformation
• Feature Learning
• Frequent Pattern Mining
• High-Content Analysis of Microscopic Images in Medicine, Biotechnology and Chemistry
• Learning and adaptive control
• Learning/adaption of recognition and perception
• Learning for Handwriting Recognition
• Learning in Image Pre-Processing and Segmentation
• Learning in process automation
• Learning of internal representations and models
• Learning of appropriate behaviour
• Learning of action patterns
• Learning of Ontologies
• Learning of Semantic Inferencing Rules
• Learning of Visual Ontologies
• Learning robots
• Mining Financial or Stockmarket Data
• Mining Images in Computer Vision
• Mining Images and Texture
• Mining Motion from Sequence
• Neural Methods
• Network Analysis and Intrusion Detection
• Nonlinear Function Learning and Neural Net Based Learning
• Real-Time Event Learning and Detection
• Retrieval Methods
• Rule Induction and Grammars
• Speech Analysis
• Statistical and Conceptual Clustering Methods: Basics
• Statistical and Evolutionary Learning
• Subspace Methods
• Support Vector Machines
• Symbolic Learning and Neural Networks in Document Processing
• Text Mining
• Time Series and Sequential Pattern Mining
• Mining Social Media

Important Dates

Deadline Paper

• Submission of papers: 15.01.2014
• Notification of acceptance: 18.03.2014
• Submission of camera-ready copy: 05.04.2014

Authors can submit their paper in long or short version:

Long Paper
The paper must be formatted in the Springer LNCS format. They should have at most 15 pages. Papers will be reviewed by the program committee. Accepted long papers will appear in the proceedings book "Machine Learning and Data Mining in Pattern Recognition" published by Springer Verlag in the LNAI series. Extended versions of selected papers will be published in a special issue of an international journal after the workshop.

Short Paper
Short papers are also welcome and can be used to describe work in progress or project ideas. They should have not more than 5 pages, formatted in Springer LNCS format. Accepted short papers will be presented as poster in the poster session. They will be published in a special poster proceedings book.

Please submit the electronic version of your camera-ready paper through the  COMMENCE conference management system.
If you have any problems with the system please do not hesitate to contact info@mldm.de.

 

Data Mining Topics & Applications 2014

The European Conference on Data Mining (ECDM’14) is aimed to gather researchers and application developers from a wide range of data mining related areas such as statistics, computational intelligence, pattern recognition, databases and visualization. ECDM’14 is aimed to advance the state of the art in data mining field and its various real world applications. ECDM’14 will provide opportunities for technical collaboration among data mining and machine learning researchers around the globe.

Acceptance will be based primarily on originality, significance and quality of contribution.

Topics for this conference include, but are not limited to:

Core Data Mining Topics - Parallel and distributed data mining algorithms - Data streams mining - Graph mining - Spatial data mining - Text video, multimedia data mining - Web mining - Pre-processing techniques - Visualization - Security and information hiding in data mining

Data Mining Applications - Databases, - Bioinformatics, - Biometrics - Image analysis - Financial modeling - Forecasting - Classification - Clustering - Social Networks - Educational data mining

For further details please contact the publicity chair - secretariat@datamining-conf.org

This is a blind peer-reviewed conference.

Important Dates
- Submission Deadline: 31 January 2014
- Notification to Authors: 3 March 2014
- Final Camera-Ready Submission and Early Registration: until 31 March 2014
- Late Registration: after 31 March 2014
- Conference: 15 - 17 July, 2014

Image processing using Data mining

Introduction to data mining images

Image processing is one of those things people are still much better at than computers.  Take this set of cats:



Just at a glance, you can easily tell the difference between the cartoon animals and the photographs.  You can tell that the hearts in the top left probably don’t belong, and that Odie is tackling Garfield in the top right.  The human brain does this really well on small datasets.

But what if we had thousands, millions, or even billions of images?  Could we make an image search engine, where I give it a picture of an animal and it says what type it is?  Could we make it automatically find patterns that people miss?

Yes!  This post is the beginning of a series about how.  Finding patterns in large databases of images is still an active research area, and these posts will hopefully make those results more accessible.  The current research still isn’t perfect, but it’s probably much better than you’d guess.

The “black box” framework
There are three basic steps in data mining images:
STEP 1: Create the “black box”
STEP 2: Cluster
STEP 3: Run queries
That’s it!
… well … sort of …

There are many different algorithms that can be used at each step.  Which ones you decide to use will depend on the type of information you’re mining from the images.  The rest of this post gives a high level overview of how each of these steps works, and later posts will focus on specific implementations for each step.

STEP 1: Creating the black box

The black box defines the “distance” between two images.  The smaller the distance, the more similar the images are.  For example:


Garfield is very similar to himself, that’s why Box A gives him a low score–nearly zero.  Odie is not very similar to Garfield, but he’s a lot closer than a palm tree.  The specific numbers outputted don’t matter.  All that matters is the ordering created by those numbers.  In this case:


Of course, if we compare against a different image, we will probably get a different ordering.
Likewise, we can get different orderings with a different black box.  Let’s imagine that Box A was designed to determine if two pictures are of the same type of animal.  If we test it on some new input, we might get:


Notice that Box A thinks the real cat is more similar to Garfield than Odie is.  Now let’s consider another black box.  Imagine Box B is designed to see if two images were drawn in a similar style.  Box B might give the following:


Now, Odie is similar to Garfield (they’re both drawn by Jim Davis), but the cat is no longer similar to Garfield (because it’s a photograph).  Box B gives the opposite results of box A.
Creating a good black box is the hardest part of data mining images.  Most research is dedicated to this area, and most of this series will be focused on evaluating the performance of different black boxes.  Which ones are good depends on your dataset and what information you’re trying to extract.  Some general categories of black boxes we’ll look at are:

1. Histogram analysis (a simple technique that can be surprisingly effective on colored input)
2. Converting images into a time series (for analyzing the shapes of rigid objects, e.g. fruit)
3. Creating shock graphs (for analyzing the shapes of non-rigid objects, e.g. animals)
4. Komolgorov comlexity of the images (for comparing an image’s textures)

But first, let’s take a closer look at what makes a black box good.

Properties of a good black box


There are two more aspects of black boxes we must look at.  First, every black box will be sensitive to certain features of an image and invariant to others.  In the examples below, Box C is sensitive to shape, but invariant to color.  Box D is sensitive to color, but invariant to shape.




Most black box algorithms contain both sensitivities and invariances.  These are the properties you will use to decide which black box is best for your application.
Second, a black box is a metric and as such must satisfy four criteria:

1. distance(x, y) ≥ 0     (non-negativity)
2. distance(x, y) = 0   if and only if   x = y     (identity of indiscernibles)
3. distance(x, y) = distance(y, x)     (symmetry)
4. distance(x, z) ≤ distance(x, y) + distance(y, z)     (subadditivity / triangle inequality).

If you don’t understand these criteria, don’t worry too much.  All the black boxes we’ll look at in the rest of this series will satisfy these criteria automatically.

STEP 2: Cluster the images

Clustering is much easier than designing the black box.  Clustering algorithms are used in many fields, so they have received much more attention.  Some clustering algorithms commonly used are:

1. Support vector machines
2. K-means
3. Neural networks
4. Hierarchical clustering (i.e. Dendrograms)

There are many more as well.  In general, you can use whatever clustering algorithm you want.  When developing an application, most people will try several and pick whichever one happens to work the best for their data.

Here’s an example clustering of our cat data using Black Box A (i.e. by what the picture is of):



We’ve created three clusters.  The red cluster contains hearts, the white cluster contains cats, and the blue cluster is an anomaly.  It contains both a cat and a dog, and there is no easy way to separate them.  If we had used a hierarchical classifier, the “contains cats and dogs cluster” might be a sub-cluster of the “contains cats cluster.”

Here’s the same data clustered using Black Box B (i.e. by how the picture is drawn):



Now we have only two clusters: the white cluster contains cartoons, and the red cluster contains photographs.One last note.  Most of the CPU work gets done during this step.  On large datasets, clustering can take hours to months depending on the algorithm and the speed of the black box.  There are many tricks for speeding up clustering, which will take a look at in later posts.


STEP 3: Run your query

Queries are fairly easy once the ground work is set up with the black box and clustering. Sometimes, all you want to know is how STEP 2 clustered your input.  For example, you could query “how many types of animals are in this dataset?”  The answer would just be the number of clusters using Box A.  Typically, however, your query we will supply the database with an image and find similar images.


If we’ve done steps 1 and 2 well, this should take only seconds even when the database contains millions of images.  Of course, it’s not always possible to do steps 1 and 2 well enough to make this happen.  Later posts may cover some new techniques for speeding up the querying process.

The Rest of the Series

So far, we’ve seen that the black box framework for image data mining is very simple:

STEP 1: Create the “black box”
STEP 2: Cluster
STEP 3: Run queries

The tricky part is putting the right algorithm in each step.  In the rest of the series, we’ll look at a few different black boxes, and show how to efficiently combine them with a clustering algorithm.  The different types of black boxes are the most interesting part of image mining, so we will focus on that first.