FWD:用EXCEL計算student's t test之p value

http://bioant.blogspot.com.au/2009/12/excelstudent-t-testp-value.html

在數值統計分析應用上，對於兩組數據差異程度判定，通常是以期望值p valuse來做為判斷的依據
p<0.05表示兩組數據之間達到統計分析上"顯著差異"的標準
p<0.001表示兩組數據之間達到統計分析上"非常顯著差異"的標準

計算這一類東西，大多數人會聽從學校的教育
找統計學軟體來安裝，而其實excel就有內建這功能了

只用的函數名稱為TTEST(array1,array2,tails,type)
(1)array1表示第1組數據，寫法可分直接寫和excel座標
A B
1 1 11
2 2 12
3 3 13
4 4 14
5 5 15
6 6 16
7 7 17
8 8 18
9 9 19
10 10 20

ex：第1組數據如果是1~10則可以用{1;2;3;4;5;6;7;8;9;10}表示 ，或是描述1~10所在的位置，可用A1:A10表示，同理array2可為{11;12;13;14;15;16;17;18;19;20}或B1:B10

(2)tails則是告訴程式是單尾分佈(數值只為+或是只為-)、還是雙尾分佈(+-值都有)，單尾則寫1、雙尾寫2
(3)type則是檢定得種類，1是成偶(兩組同一母體相依的數據，如治療前後)、2是具有相同變異數(通常只來自於同一母體的2樣本)、3是具有不同變異數

所以以上資料是具有相同變異數的單尾分佈資料，計算公式為=ttest(A1:A10,B1:B10,1,2)，p value為0.000000375
因為p<0.001所以兩組數據差異非常顯著
--------------------------------------------------------------- 2011.10.17補充
應鄉民要求，對TTEST的檢定種類(type)再做一些說明
(1)是成對資料：相同的100個人，減肥前&減肥後體重；吃藥前&吃藥後血壓
(2、3)相同變異數與不同變異數的資料：如：大人跟小孩吃東西的多寡；男生女生睡javascript:void(0)眠的長短。選擇的標準是"變異數"
若FTEST(array1,array2)>=0.05，表示兩組數據的變異數相同，請使用(2)
若FTEST(array1,array2)<0.05，表示兩組數據的變異數不同，請使用(3)

我最討厭專有名詞啦，大家會用就好
以前type寫錯成3的事我一定不會承認的啦(((诶~好像哪怪怪的

Teradata warehouse introduction

http://www.info.teradata.com/eDownload.cfm?itemid=062330006&redirectfromfle=y

A single Teradata system can support a maximum of 16,384 vprocs (virtual processor). The maximum number of vprocs per node can be as high as 128, but is typically between 6 and 12

The AMP is the heart of the Teradata Database. The AMP is a vproc that controls the
management of the Teradata Database and the disk subsystem, with each AMP being assigned to a vdisk.

Vprocs provide the parallel environment that enables the Teradata Database to run on SMP and MPP systems. Vprocs come in two types:
• Access Module Processors (AMPs) ( manage a portion of disk
• Parsing Engines (PEs). (decompose sql and return answer (dispatcher; parser;opti;)

Query are scattered to BYNET linked CLIQUE (Fully inter-linked nodes and arrays), AMP get instruction and return data (3 steps: LOCK for serialization and consistency;

************************************************
Teradata Training Program
Total Training Duration: 3 PD ( 24 Hour)

- Basics (2Hours)
- Architecture (3 hours)
- SQL (4 Hours)
- ETL Utilities (3 Hours)
- Case Study (12 Hours)


Basics:

- Basic Architecture of Teradata
- The uniqueness of Teradata SQL and to know different functions available in Teradata.
- Different Utilities (FastLoad, FastExport, MultiLoad, Tpump,Teradata Warehouse Miner) in Teradata.
- How Teradata is ideal for Datawarehouse application.


Architecture:

Hardware components:
The hardware that supports Teradata RDBMS software is based on off the-shelf Symmetric Multiprocessing (SMP) technology. Hardware Components are:

• SMP Systems
• MPP Systems
• BYNET
• VPROCS
- Parsing Engine (PE )
- Access Module Processor (AMP)

Software Architecture:
• Parallel Database Extensions (PDE)
• Teradata File System
• Trusted Parallel Applications (TPA )
• Virtual Processors - PE and AMP
• PERM Space
• SPOOL Space



SQL:
- Teradata DDL(create, Drop, Alter)
- Teradata DML(Select, Insert,Update, Delete),
- Teradata DCL(Grant, Revoke).
- Teradata extensions to SQL like HELP, SHOW, EXPLAIN, CREATE MACRO, REPLACE MACRO.
- Primary index / Secondary Index
- SET operators(UNION,INTERSECT,EXCEPT, MINUS)


ETL Utilities:

Fload: This utility provides us with facility for loading new tables onto Teradata database

Fast export: This utility exports a large volume of data from Teradata to a host file or user-written application.

Multiload: This utility supports INSERT,UPDATE, DELETE and UPSERTS typically with batch inputs from a host file.

TPump: This utility allows the real time update from transactional systems into warehouse.


Case Study:

Administration:
- To create a user , database and understand the hierarchy of user and database.
- To create a table with columns having datatype integer, char, decimal, date and having one primary index.
- To drop a table in a database or a user.

SQL:
- To use SELECT statement with ORDERBY and GROUPBY clause.
- To use inner join, LEFT Outer Join, Right Outer Join and Full Outer Join on two tables.
- To use CASE, COALESCE, NULLIF, IFNULL
- To use set operators(UNION,INTERSECT,EXCEPT)

Fast Load:
- To create a empty table in TeraData and prepare a flat file containing data. Prepare a script for loading the data from flat file into the empty table.

Fast Export:

- To use the FastExport to create an export file that contains one record for each transaction . Columns from two different tables has to be joined inorder to create the export file.

MultiLoad:
- To use Multiload to delete rows from your three tables. An input file will be created which will contain a control letter(A-Accounts, C-customer, T-trans) followed by a primary index value for the appropriate table

TPUMP:
- Use FastExport to export the records into a flat file. Prepare Tpump script which performs UPSERT operation on another table.Validate your results.

一百万美元等你来挖

http://www.netflixprize.com//community/viewtopic.php?id=147

该公司出100万美元，悬赏能突破cinematch电影推荐算法，或5万美元的年度进步奖

有人推理出是因为 choicestream（如今该公司用的程序）的授权费每年超过100万美元 :P

以下是几个有趣的技术讨论
>

Because MySQL is basically a file-system and not a true database, it is faster

Like MySQL and PostGres it's free. Except Oracle XE is crippled in that it will only allow databases up to 4GB, will only use 1GB of RAM, and will only use 1 processor (if you have more than one). If your application fits within those limits then it *might* be faster than MySQL, but probably only if you're using it in a transactional sense. MySQK with MyISAM tables should be very, very fast. Especially if you have a machine with specs that exceed the limitations placed on Oracle XE