Showing posts with label Data science. Show all posts
Showing posts with label Data science. Show all posts

Sunday, March 18, 2018

Top 10 Technology Trends for 2018: IEEE Computer Society Predicts the Future of Tech



The top 10 technology trends predicted to reach adoption in 2018 are:
  1. Deep learning (DL)Machine learning (ML) and more specifically DL are already on the cusp of revolution. They are widely adopted in datacenters (Amazon making graphical processing units [GPUs] available for DL, Google running DL on tensor processing units [TPUs], Microsoft using field programmable gate arrays [FPGAs], etc.), and DL is being explored at the edge of the network to reduce the amount of data propagated back to datacenters. Applications such as image, video, and audio recognition are already being deployed for a variety of verticals. DL heavily depends on accelerators (see #9 below) and is used for a variety of assistive functions (#s 6, 7, and 10). 

  2. Digital currenciesBitcoin, Ethereum, and newcomers Litecoin, Dash, and Ripple have become commonly traded currencies. They will continue to become a more widely adopted means of trading. This will trigger improved cybersecurity (see #10) because the stakes will be ever higher as their values rise. In addition, digital currencies will continue to enable and be enabled by other technologies, such as storage (see #3), cloud computing (see B in the list of already adopted technologies), the Internet of Things (IoT), edge computing, and more.

  3. Blockchain. The use of Bitcoin and the revitalization of peer-to-peer computing have been essential for the adoption of blockchain technology in a broader sense. We predict increased expansion of companies delivering blockchain products and even IT heavyweights entering the market and consolidating the products. 

  4. Industrial IoT. Empowered by DL at the edge, industrial IoT continues to be the most widely adopted use case for edge computing. It is driven by real needs and requirements. We anticipate that it will continue to be adopted with a broader set of technical offerings enabled by DL, as well as other uses of IoT (see C and E). 

  5. Robotics. Even though robotics research has been performed for many decades, robotics adoption has not flourished. However, the past few years have seen increased market availability of consumer robots, as well as more sophisticated military and industrial robots. We predict that this will trigger wider adoption of robotics in the medical space for caregiving and other healthcare uses. Combined with DL (#1) and AI (#10), robotics will further advance in 2018. Robotics will also motivate further evolution of ethics (see #8).

  6. Assisted transportation. While the promise of fully autonomous vehicles has slowed down due to numerous obstacles, a limited use of automated assistance has continued to grow, such as parking assistance, video recognition, and alerts for leaving the lane or identifying sudden obstacles. We anticipate that vehicle assistance will develop further as automation and ML/DL are deployed in the automotive industry.

  7. Assisted reality and virtual reality (AR/VR)Gaming and AR/VR gadgets have grown in adoption in the past year. We anticipate that this trend will grow with modern user interfaces such as 3D projections and movement detection. This will allow for associating individuals with metadata that can be viewed subject to privacy configurations, which will continue to drive international policies for cybersecurity and privacy (see #10).

  8. Ethics, laws, and policies for privacy, security, and liability. With the increasing advancement of DL (#1), robotics (#5), technological assistance (#s 6 and 7), and applications of AI (#10), technology has moved beyond society's ability to control it easily. Mandatory guidance has already been deeply analyzed and rolled out in various aspects of design (see the IEEE standards association document), and it is further being applied to autonomous and intelligent systems and in cybersecurity. But adoption of ethical considerations will speed up in many vertical industries and horizontal technologies.

  9. Accelerators and 3D. With the end of power scaling and Moore's law and the shift to 3D, accelerators are emerging as a way to continue improving hardware performance and energy efficiency and to reduce costs. There are a number of existing technologies (FPGAs and ASICs) and new ones (such as memristor-based DPE) that hold a lot of promise for accelerating application domains (such as matrix multiplication for the use of DL algorithms). We predict wider diversity and broader applicability of accelerators, leading to more widespread use in 2018.

  10. Cybersecurity and AI. Cybersecurity is becoming essential to everyday life and business, yet it is increasingly hard to manage. Exploits have become extremely sophisticated and it is hard for IT to keep up. Pure automation no longer suffices and AI is required to enhance data analytics and automated scripts. It is expected that humans will still be in the loop of taking actions; hence, the relationship to ethics (#8). But AI itself is not immune to cyberattacks. We will need to make AI/DL techniques more robust in the presence of adversarial traffic in any application area.
Existing Technologies: We did not include the following technologies in our top 10 list as we assume that they have already experienced broad adoption:
A.      Data science
B.      "Cloudification"
C.      Smart cities
D.      Sustainability
E.       IoT/edge computing

IEEE-CS technical contributors include Erik DeBenedictis, Sandia National Laboratories; Fred Douglis, systems researcher and member of IEEE-CS Board of Governors; David Ebert, professor, Purdue University; Paolo Faraboschi, Hewlett Packard Enterprise Fellow; Eitan Frachtenberg, data scientist; Phil Laplante, professor, Penn State University; and Dejan Milojicic, Hewlett Packard Enterprise Distinguished Technologist and IEEE Computer Society past president.  

Monday, December 12, 2016

Top Analytics, Data Science software

 

 

R, Python Duel As Top Analytics, Data Science software – KDnuggets 2016 Software Poll Results

R remains the leading tool, with 49% share, but Python grows faster and almost catches up to R. RapidMiner remains the most popular general Data Science platform. Big Data tools used by almost 40%, and Deep Learning usage doubles.  

The poll got tremendous participation from analytics and data science community and vendors, attracting 2,895 voters, who chose from a record number of 102 different tools.

R remains the leading tool, with 49% share (up from 46.9% in 2015), but Python usage grew faster and it almost caught up to R with 45.8% share (up from 30.3%). RapidMiner remains the most popular general platform for data mining/data science, with about 33% share. Notable tools with the most growth in popularity include Dato, Dataiku, MLlib, H2O, Amazon Machine Learning, scikit-learn, and IBM Watson.

The increased choice of tools is reflected in wider usage. The average number of tools used was 6.0, vs 4.8 in 2015.

The usage of Hadoop/Big Data tools grew to 39%, up from 29% in 2015 (and 17% in 2014), driven by Apache Spark, MLlib (Spark Machine Learning Library) and H2O.
The participation by region was: US/Canada (40%), Europe (39%), Asia (9.4%), Latin America (5.8%), Africa/MidEast (2.9%), Australia/NZ (2.2%).

Top Analytics/Data Science Tools

Next table has the top 10 most popular tools in 2016 poll
Tool2016
% share
% change% alone
R49%+4.5% 1.4%
Python45.8%+51% 0.1%
SQL35.5%+15% 0%
Excel33.6%+47% 0.2%
RapidMiner32.6%+3.5% 11.7%
Hadoop22.1%+20% 0%
Spark21.6%+91% 0.2%
Tableau18.5%+49% 0.2%
KNIME18.0%-10%4.4%
scikit-learn17.2%+107% 0%

In this table 2016 % share is % of voters who used this tool, % change is the change in share vs 2015 poll, and % alone is the percent of voters who used only the reported tool among all voters who used that tool. E.g. 4.4% of KNIME voters reported using only KNIME and nothing else. We note a decrease in such lone voting, with only 9 tools having 5% or more lone votes.

Top10 Analytics Data Science Software 2016
Fig 1: KDnuggets Analytics/Data Science 2016 Software Poll: top 10 most popular tools in 2016

Compared to 2015 KDnuggets Analytics/Data Science Poll results, the only newcomer in top 10 was scikit-learn, displacing SAS.

Tools with the highest growth (among tools with at least 15 users in 2015) were
Tool% change2016 %share2015 %share
Dato377%2.4%0.5%
Dataiku292%7.8%2.0%
MLlib253%11.6%3.3%
H2O233%6.7%2.0%
Amazon Machine Learning171%1.9%0.7%
scikit-learn107%17.2%8.3%
IBM Watson99%4.2%2.1%
Splunk/ Hunk98%2.2%1.1%
Spark91%21.6%11.3%
Scala79%6.2%3.5%


This year, 86% of voters used commercial software and 75% used free software. About 25% used only commercial software, and 13% used only open source/free software. A majority of 61% used both free and commercial software, similar to 64% in 2015.

New (in this poll) tools that received at least 1% share votes in 2016 were
  • Anaconda, 16%
  • Microsoft other ML/Data Science tools, 1.6%
  • SAP HANA, 1.2%
  • XLMiner, 1.2%
Among tools with at least 15 votes in 2015, the largest decline in 2016 was for the tools below, which includes probably a combination of decline of popularity for free tools like F# and lack of a voter drive for some of commercial tools this year.
  • Ayasdi, down 85%, to 0.3% share from 2.0%
  • Actian, down 83%, to 0.3% share from 2.0%
  • Datameer, down 52%, to 0.4% share from 0.9%
  • SAP Analytics, down 51%, to 1.5% share from 3.0%
  • SAS Enterprise Miner, down 49%, to 5.6% from 10.9%
  • Alteryx, down 46%, to 3.0% share from 5.6%
  • F#, down 42%, to 0.4% share from 0.7%
  • TIBCO Spotfire, down 36%, to 2.8% share from 4.3%
  • JMP, down 36%, to 2.0% share from 3.1%

Hadoop/Big Data Tools

The usage of Hadoop/Big Data tools grew to 39%, up from 29% in 2015 and 17% in 2014), driven mainly by big growth in Apache Spark, MLlib (Spark Machine Learning Library) and H2O, which we included among Big Data tools.

Here are the Big Data tools and their share in 2016, 2015, and %change.
Tool2016
%Share
2015
%share
% change
Hadoop22.1%18.4%+20.5%
Spark21.6%11.3%+91%
Hive12.4%10.2%+21.3%
MLlib11.6%3.3%+253%
SQL on Hadoop tools7.3%7.2%+1.6%
H2O6.7%2.0%+234%
HBase5.5%4.6%+18.6%
Apache Pig4.6%5.4%-16.1%
Apache Mahout2.6%2.8%-7.2%
Dato2.4%0.5%+338%
Datameer0.4%0.9%-52.3%
Other Hadoop/HDFS-based tools4.9%4.5%+7.5%

Deep Learning Tools

For the second year KDnuggets poll include Deep Learning Tools. This year, 18% of voters used Deep Learning tools, doubling the 9% in 2015.

Google Tensorflow jumped to first place, displacing last year leader Theano/Pylearn2 ecosystem.

Top tools:
  • Tensorflow, 6.8%
  • Theano ecosystem (including Pylearn2), 5.1%
  • Caffe, 2.3%
  • MATLAB Deep Learning Toolbox, 2.0%
  • Deeplearning4j, 1.7%
  • Torch, 1.0%
  • Microsoft CNTK, 0.9%
  • Cuda-convnet, 0.8%
  • mxnet, 0.6%
  • Convnet.js, 0.3%
  • darch, 0.1%
  • Nervana, 0.1%
  • Veles, 0.1%
  • Other Deep Learning Tools, 3.7%
The Deep Learning field is still in the beginning of its journey, as we see by the large number of options.

Programming Languages

Python, Java, Unix tools, Scala grew in popularity, while C/C++, Perl, Julia, F#, Clojure, and Lisp declined.

Here are the programming languages sorted by popularity.
  • Python, 45.8% share (was 30.3%), 51% increase
  • Java, 16.8% share (was 14.1%), 19% increase
  • Unix shell/awk/gawk 10.4% share (was 8.0%), 30% increase
  • C/C++, 7.3% share (was 9.4%), 23% decrease
  • Other programming/data languages, 6.8% share (was 5.1%), 34.1% increase
  • Scala, 6.2% share (was 3.5%), 79% increase
  • Perl, 2.3% share (was 2.9%), 19% decrease
  • Julia, 1.1% share (was 1.1%), 1.6% decrease
  • F#, 0.4% share (was 0.7%), 41.8% decrease
  • Clojure, 0.4% share (was 0.5%), 19.4% decrease
  • Lisp, 0.2% share (was 0.4%), 33.3% decrease