在企业里面从事大数据相关的工作到底需要掌握哪些知识呢？我认为需要从两个角度来看：一个是技术；一个是业务。技术上主要涉及到概率和数理统计，计算机系统、算法和编程等；而业务的角度呢则是因公司业务的不同而异。对于从事大数据的工程人员来说，需要学会使用数据挖掘方法在计算机系统和编程工具的帮助下解决实际的问题，这样才能够在海量数据中挖掘出业务增长的助推剂，才能在激烈的市场竞争中为企业创造更多的价值。
因为业务会因公司的不同而不同，但是技术点是想通的。我在这里简单总结了一下大数据相关工程人员需要掌握的技术相关知识点。主要涉及到数据库、数据仓库、编程、分布式系统、hadoop生态系统相关、数据挖掘和机器学习相关的基础知识点。当然我这里列出来的应该是一个team的人员汇集在一起所具备的，每个人会因在团队中的角色不同而有所侧重。在此剖砖引玉，欢迎大家发表意见。
topic content key points reference
db/oltp & dw/olap database/oltp basic the relational model, sql, index/secondary index, inner join/left join/right join/full join, transaction/acid ramakrishnan, raghu, and johannes gehrke. database management systems.
database internal & implementation architecture, memory management, storage/b+ tree, query parse /optimization/execution, hash join/sort-merge join
distributed and parallel database sharding, database proxy
data warehouse/olap materialized views, etl, column-oriented storage, reporting, bi tools
basic programming programming language java, python (pandas/numpy/scipy/scikit-learn), sql, functional programming, r/sas/spss wes mckinney. python for data analysis: agile tools for real world data.
os linux
db & dw system mysql/ hive/impala
text format and process json/xml, regex
tool git/svn, maven
distributed system & hadoop ecosystem & nosql distributed system principal theory cap theorem, rpc (protocol buffer/thrift/avro), zookeeper, metadata management (hcatalog)
distributed storage & computing framework & resource management hadoop/hdfs/mapreduce/yarn tom white. hadoop : the definitive guide.donald miner, adam shook. mapreduce design patterns : building effective algorithm and analytics for hadoop and other systems.
sql on hadoop data (log) acquisition/integration/fusion, normalization, feature extraction sqoop, flume/scribe/chukwa,serde edward capriolo, dean wampler, jason rutherglen. programming hive.
query & in-database analytics hive, impala, udf/udaf
large scale data mining & machine learning framework spark/mlbase, mr/mahout
streaming process storm
nosql hbase/cassandra (column oriented database) lars george. hbase: the definitive guide.
mongodb (document database)
neo4j (graph database)
redis (cache)
data mining & machine learning dm & ml basic numerical/categorical variable, training/test data, over fitting, bias/variance, precision/recall, tagging
statistic data exploration (mean, median/range/standard deviation/variance/histogram), continues distributions (normal/ poisson/gaussian), covariance, correlation coefficient, distance and similarity computing, bayes theorem, monte carlo method, hypothesis testing
supervised learning classifier, boosting, prediction, regression analysis han, jiawei,micheline kamber, and jian pei.?data mining: concepts and techniques.
unsupervised learning cluster, deep learning
collaborative filtering item based cf, user based cf
algorithm classifier decision trees, knn (k-nearest neighbor), svm (support vector machines), svd (singular value decomposition), na?ve bayes classifiers, neural networks,
regression linear regression, logistic regression, ranking, perception
cluster hierarchical cluster, k-means cluster, spectral cluster
dimensionality reduction pca (principal component analysis), lda (linear discriminant analysis), mds (multidimensional scaling)
text mining & information retrieval corpus, term document matrix, term frequency & weight, association rules, market based analysis, vocabulary mapping, sentiment analysis, tagging, pagerank, vsm (vector space model), inverted index jimmy lin and chris dyer. data-intensive text processing with mapreduce.
原文地址：大数据工程人员知识图谱, 感谢原作者分享。