Example Source: Advanced Analytics with Spark

source: https://github.com/sryza/aas

Advanced Analytics with Spark
목차:
1. 오디오스크로블로 데이터셋으로 음악 추천하기
2. 의사 결정 나무로 산림 식샐 분포 예측하기
3. K-평균 군집화로 네트워크 이상 감지하기
4. 숨은 의미 분석으로 위키 백과 이해하기
5. 그래프엑스로 동시 발생 네트워크 분석하기
6. 뉴욕 택시 운행 데이터로 위치 및 시간 데이터 분석하기
7. 몬테카를로 시뮬레이션으로 금융 리스크 추정하기
8. BDG 프로젝트와 유전체학 데이터 분석하기
9. PySpark와 Thunder로 신경 영상 데이터 분석하기.

Code to accompany Advanced Analytics with Spark from O'Reilly Media

•  291 commits
•  8 branches
•  2 releases
•  7 contributors

1.  Scala 91.8%
2.  Python 4.9%
3.  R 2.8%
4.  Shell 0.5%

ScalaPythonRShell

Find file

Latest commit e8754e0 2 days ago sryza committed on GitHub Fix LSA issues and harmonize with the text (#104)

	ch02-intro	Remove ref to Juliet as an author for 2nd edition	2 months ago
	ch03-recommender	Tiny change to match text changes from review	26 days ago
	ch04-rdf	Use getEstimatorParamMaps per reviewer suggestion	2 months ago
	ch05-kmeans	Remove ref to Juliet as an author for 2nd edition	2 months ago
	ch06-lsa	Fix LSA issues and harmonize with the text (#104)	2 days ago
	ch07-graph	Remove ref to Juliet as an author for 2nd edition	2 months ago
	ch08-geotime	Remove ref to Juliet as an author for 2nd edition	2 months ago
	ch09-risk	Remove kernel density implementation (#102)	3 days ago
	ch10-genomics	Remove ref to Juliet as an author for 2nd edition	2 months ago
	ch11-neuro	Remove ref to Juliet as an author for 2nd edition	2 months ago
	simplesparkproject	Use Spark 2.1 in example project	12 days ago
	.gitignore	Define intro module	2 years ago
	.travis.yml	Update deps, plugins (#91)	3 months ago
	LICENSE	Remove ref to Juliet as an author for 2nd edition	2 months ago
	README.md	Also revert author order to match book	a month ago
	pom.xml	Fix last-second typo	a month ago

 README.md

Advanced Analytics with Spark Source Code

Code to accompany Advanced Analytics with Spark, by Sandy Ryza, Uri Laserson, Sean Owen, and Josh Wills.

1st Edition (current)

The source to accompany the 1st edition may be found in the 1st-edition branch.

2nd Edition (coming H1 2017)

The source to accompany the 2nd edition is found in this, the default master branch.

Build

Apache Maven 3.2.5+ and Java 8+ are required to build. From the root level of the project, run mvn package to compile artifacts into target/ subdirectories beneath each chapter's directory.

Data Sets

• Chapter 2: https://archive.ics.uci.edu/ml/machine-learning-databases/00210/
• Chapter 3: http://www-etud.iro.umontreal.ca/~bergstrj/audioscrobbler_data.html
• Chapter 4: https://archive.ics.uci.edu/ml/machine-learning-databases/covtype/
• Chapter 5: https://kdd.ics.uci.edu/databases/kddcup99/kddcup99.html (do not use http://www.sigkdd.org/kdd-cup-1999-computer-network-intrusion-detection as the copy has a corrupted line)
• Chapter 6: https://dumps.wikimedia.org/enwiki/latest/enwiki-latest-pages-articles-multistream.xml.bz2
• Chapter 7: ftp://ftp.nlm.nih.gov/nlmdata/sample/medline/ (*.gz)
• Chapter 8: http://www.andresmh.com/nyctaxitrips/
• Chapter 9: (see ch09-risk/data/download-all-symbols.sh script)
• Chapter 10: ftp://ftp.ncbi.nih.gov/1000genomes/ftp/phase3/data/HG00103/alignment/HG00103.mapped.ILLUMINA.bwa.GBR.low_coverage.20120522.bam
• Chapter 11: https://github.com/thunder-project/thunder/tree/v0.4.1/python/thunder/utils/data/fish/tif-stack