Hadoop 2.0.4-alpha 版发布
Hadoop 2.0.4-alpha 版发布了,此版本相对于Hadoop 2.0.3-alpha做出的一些改进如下:
Changes since Hadoop 2.0.3-alpha
YARN-470. Major bug reported by Hitesh Shah and fixed by Siddharth Seth (nodemanager)
Support a way to disable resource monitoring on the NodeManagerCurrently, the memory management monitor's check is disabled when the maxMem is set to -1. However, the maxMem is also sent to the RM when the NM registers with it ( to define the max limit of allocate-able resources ). We need an explicit flag to disable monitoring to avoid the problems caused by the overloading of the max memory value.
YARN-449. Blocker bug reported by Siddharth Seth and fixed by
HBase test failures when running against Hadoop 2Post YARN-429, unit tests for HBase continue to fail since the classpath for the MRAppMaster is not being set correctly. Reverting YARN-129 may fix this, but I'm not sure that's the correct solution. My guess is, as Alexandro pointed out in YARN-129, maven classloader magic is messing up java.class.path.
YARN-443. Major improvement reported by Thomas Graves and fixed by Thomas Graves (nodemanager)
allow OS scheduling priority of NM to be different than the containers it launchesIt would be nice if we could have the nodemanager run at a different OS scheduling priority than the containers so that you can still communicate with the nodemanager if the containers out of control. On linux we could launch the nodemanager at a higher priority, but then all the containers it launches would also be at that higher priority, so we need a way for the container executor to launch them at a lower priority. I'm not sure how this applies to windows if at all.
YARN-429. Blocker bug reported by Siddharth Seth and fixed by Siddharth Seth (resourcemanager)
capacity-scheduler config missing from yarn-test artifactminiyarncluster and minimrcluster are unusable by downstream projects with the 2.0.3-alpha release, since the capacity-scheduler configuration is missing from the test artifact. hadoop-yarn-server-tests-3.0.0-snapshot-tests.jar should include the default capacity-scheduler configuration. also, this doesn't need to be part of the default classpath - and should be moved out of the top level directory in the dist package.
MAPREDUCE-5117. Blocker bug reported by Roman Shaposhnik and fixed by Siddharth Seth (security)
With security enabled HS delegation token renewer failsMAPREDUCE-5094. Major bug reported by Siddharth Seth and fixed by Siddharth Seth
Disable mem monitoring by default in MiniMRYarnClusterMAPREDUCE-5088. Blocker bug reported by Roman Shaposhnik and fixed by Daryn Sharp
MR Client gets an renewer token exception while Oozie is submitting a jobMAPREDUCE-5083. Major bug reported by Siddharth Seth and fixed by Siddharth Seth (mrv2)
MiniMRCluster should use a random component when creating an actual clusterCommitted to branch-2.0.4. Modified changes.txt in trunk, branch-2 and branch-2.0.4 accordingly.
MAPREDUCE-5053. Major bug reported by Robert Parker and fixed by Robert Parker
java.lang.internalerror from decompression codec cause reducer to failMAPREDUCE-5023. Critical bug reported by Kendall Thrapp and fixed by Ravi Prakash (jobhistoryserver , webapps)
History Server Web Services missing Job CountersMAPREDUCE-5006. Major bug reported by Alejandro Abdelnur and fixed by Sandy Ryza (contrib/streaming)
streaming tests failingMAPREDUCE-4549. Blocker bug reported by Robert Joseph Evans and fixed by Robert Joseph Evans (mrv2)
Distributed network iis7站长之家 conflicts breaks backwards compatabilityHDFS-4649. Blocker bug reported by Daryn Sharp and fixed by Daryn Sharp (namenode , security , webhdfs)
Webhdfs cannot list large directoriesHDFS-4646. Minor bug reported by Jagane Sundar and fixed by (namenode)
createNNProxyWithClientProtocol ignores configured timeout valueHDFS-4581. Major bug reported by Rohit Kochar and fixed by Rohit Kochar (datanode)
DataNode#checkDiskError should not be called on network errorsHDFS-4577. Major sub-task reported by Daryn Sharp and fixed by Daryn Sharp (webhdfs)
Webhdfs operations should declare if authentication is requiredHDFS-4571. Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (webhdfs)
WebHDFS should not set the service hostname on the server sideHDFS-4567. Major sub-task reported by Daryn Sharp and fixed by Daryn Sharp (webhdfs)
Webhdfs does not need a token for token operationsHDFS-4566. Major sub-task reported by Daryn Sharp and fixed by Daryn Sharp (webhdfs)
Webdhfs token cancelation should use authenticationHDFS-4560. Major sub-task reported by Daryn Sharp and fixed by Daryn Sharp (webhdfs)
Webhdfs cannot use tokens obtained by another userHDFS-4548. Blocker sub-task reported by Daryn Sharp and fixed by Daryn Sharp
Webhdfs doesn't renegotiate SPNEGO tokenHDFS-3344. Major bug reported by Tsz Wo (Nicholas), SZE and fixed by Kihwal Lee (namenode)
Unreliable corrupt blocks counting in TestProcessCorruptBlocksHADOOP-9471. Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)
hadoop-client wrongfully excludes jetty-util JAR, breaking webhdfsHADOOP-9467. Major bug reported by Chris Nauroth and fixed by Chris Nauroth (metrics)
Metrics2 record filtering (.record.filter.include/exclude) does not filter by nameHADOOP-9444. Blocker bug reported by Konstantin Boudnik and fixed by Roman Shaposhnik (conf)
$var shell substitution in properties are not expanded in hadoop-policy.xmlHADOOP-9406. Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (build)
hadoop-client leaks dependency on JDK tools jarHADOOP-9405. Minor bug reported by Andrew Wang and fixed by Andrew Wang (test , tools)
TestGridmixSummary#testExecutionSummarizer is brokenHADOOP-9399. Minor bug reported by Todd Lipcon and fixed by Konstantin Boudnik (build)
protoc maven plugin doesn't work on mvn 3.0.2Committed to 2.0.4-alpha branch
HADOOP-9379. Trivial improvement reported by Arpit Gupta and fixed by Arpit Gupta
capture the ulimit info after printing the log to the consoleHADOOP-9374. Major improvement reported by Daryn Sharp and fixed by Daryn Sharp (security)
Add tokens from -tokenCacheFile into UGIHADOOP-9301. Blocker bug reported by Roman Shaposhnik and fixed by Alejandro Abdelnur (build)
hadoop client servlet/jsp/jetty/tomcat JARs creating conflicts in Oozie & HttpFSHADOOP-9299. Blocker bug reported by Roman Shaposhnik and fixed by Daryn Sharp (security)
kerberos name resolution is kicking in even when kerberos is not configured
具体地址:
Hadoop 2.1.0-beta 版发布 Hadoop开发插件 Hadoop Studio Apache Hadoop 项目主页及介绍 Hadoop部署管理和扩展 Serengeti Hadoop 2.0.5-alpha版发布 Spring Hadoop Apache hadoop当前各个版本说明 Hadoop标准化安装工具 Cloudera 安装hadoop时出现内存不够及其它一些常见的hadoop错误解决办法 Hadoop日志聚合器 White Elephant Hadoop中MultipleOutputFormat示例代码 Hadoop的Jython封装 Happy hadoop 1.2.1 总体介绍 Hadoop集群监控工具 HTools Hadoop介绍及最新稳定版Hadoop 2.4.1下载地址及单节点安装 Hadoop和数据库数据迁移工具 Sqoop 基于Hadoop的数据挖掘框架 Hadoop 开发工具 Apache HDT Hadoop框架及各部分组件介绍 分布式系统基础架构 Hadoop Hadoop 1.2.1 单节点安装(Single Node Setup)步骤