Saturday, August 27, 2011

How to resolve the Hadoop Map Reduce Out of Memory Exception

If you ever face an out of memory exception while running a hadoop job on your cluster, consider using the following parameters in your command line :

-Dmapred.map.child.java.opts=-Xmx1G -Dmapred.reduce.child.java.opts=-Xmx1G

Those parameters allow expanding the map and reduce memory limit to 1GB. Those setting can also be specified in the mapred.reduce.child.java.opts file if you have access to it so as to avoid writing them each time you run a hadoop command

6 comments:

  1. The information which you have provided is very good and easily understood. It is very useful who is looking for hadoop Training.
    Hadoop Training in hyderabad

    ReplyDelete
  2. Thanks for sharing this informative blog. Suppose if anyone interested to learn Hadoop Training institute in Chennai, Please visit Fita Academy located at Chennai, Velachery.

    ReplyDelete
  3. Hi admin thanks for sharing informative article on hadoop technology. In coming years, hadoop and big data handling is going to be future of computing world. This field offer huge career prospects for talented professionals. Thus, taking Hadoop Training in Chennai will help you to enter big data technology.

    ReplyDelete
  4. Really i am impressed from this post....the person who created this post is a genius and knows how to keep the readers connected..

    Hadoop Certification in Chennai

    ReplyDelete
  5. Really useful information about hadoop, i have to know information about hadoop online training institutes.

    ReplyDelete