UDF Collections.
DataFu
DataFu is Linkedin's collection of Pig UDFs, which has become an Apache Incubator project. (http://sna-projects.com/datafu/)
This page describes tools that users have built on top of Pig. Some of these tools make developing Pig Latin scripts easier. Some facilitate running Pig in your environment. The tools are listed in alphabetical order.
Amazon
Amazon Elastic MapReduce makes it easy to launch Pig in interactive or batch mode in AWS. The service supports Pig via AWS Management Console, Command Line Client, and its APIs.
Eclipse Plugin (Lightweight)
http://code.google.com/p/pig-eclipse
Provides pig-latin editor in eclipse, have the feature of syntax highlighting. I just make it for interests, now it has less features than pigpen.
Elephant-Bird
http://github.com/kevinweil/elephant-bird/
Twitter's library of LZO and/or Protocol Buffer-related Hadoop InputFormats, OutputFormats, Writables, Pig LoadFuncs, HBase miscellanea, etc. The majority of these are in production at Twitter running over data every day.
RPM and Debian packages for Elephant Bird can be found at http://code.google.com/p/hadoop-gpl-packing/
Emacs Pig Latin Mode
Pygmalion
httpshttp://github.com/cloudera/piglatin-mode
A simple Emacs mode that highlights Pig Latin syntax. It's very basic, but it does make life more pleasant if you are an Emacs user.
http://sf.net/projects/pig-mode
A project to facilitate using Pig with Apache Cassandra. It mostly focuses on tabular data with static column names.
Tools that help run Pig workflows
Amazon
Amazon Elastic MapReduce makes it easy to launch Pig in interactive or batch mode in AWS. The service supports Pig via AWS Management Console, Command Line Client, and its APIsAnother emacs mode for pig, with automatic indentation and full Pig 0.2 syntax support.
HAMAKE
http://code.google.com/p/hamake/
'hamake' utility allows you to automate incremental processing of datasets stored on HDFS using Hadoop tasks written in Java or using PigLatin scripts.
...
Mortar Data
http://romainrmortardata.github.com/PigEditor/
Autocomplete, show errors in red, etc.
Mortar Data's platform-as-a-service hosts and executes jobs written in the Mortar framework or in the platform's built-in web interface. Powered in part by Amazon's Elastic MapReduce, Mortar's PaaS provides full code history (backed by GitHub), automatic job scheduling (or running them via API), and emails you about job successes/failures. It also gives you instant feedback while you're coding and executing, so you can quickly find and fix errors and bottlenecks in your code.
Mortar Framework
Mortar is an open source framework for building, organizing, testing, and deploying projects using Pig. Inspired by the project structure of web frameworks like Rails, Mortar makes Pig much easier to work with by providing great utilities, cutting out boilerplate work, and by providing a standard place for your control scripts, Pig code, UDFs, tests, and libraries.It works with Eclipse 3.7: http://romainr.github.com/PigEditor/updates/
Piglet
http://github.com/iconara/piglet
Piglet is a DSL for writing Pig Latin scripts in Ruby. Piglet aims to look like Pig Latin while allowing for things like loops and control of flow that are missing from Pig.
PigPy
http://code.google.com/p/pigpy/
A Python wrapper that helps users manage their Pig processes. It can manage many similar Pig Latin scripts, including running common root scripts and caching the results to be used in generation of the final output scripts. Pig can be run directly from PigPy, allowing users to inspect results of the Pig job and take further actions.
Editor plugins
Eclipse
Pig-Eclipse
- https://github.com/eyala/pig-eclipse
- Syntax highlighting, auto complete, javadoc-style tooltips, and open declaration (F3). Supports Pig 0.11 - 0.14. Works with Eclipses running Java 6 and newer (tested on Indigo, Juno, Kepler and Luna).
- Update site: http://github.com/eyala/pig-eclipse-update-site/raw/master
Pig Editor for Eclipse
- http://romainr.github.com/PigEditor/
Autocomplete, show errors in red, etc. Supports Pig 0.09 / 0.10. Has not been maintained for a while.
It works with Eclipse 3.7: http://romainr.github.com/PigEditor/updates/
PigPen
...
- An Eclipse plugin for Pig. In addition to standard syntax highlighting it supports graphical construction of Pig Latin scripts (box and arrow) as well as showing the user the output schema of each line in a Pig Latin script and example output of his or her script. Pig can be run directly from PigPen in either local or Map-Reduce mode. Has not been maintained for a while.
- An Eclipse plugin for Pig. In addition to standard syntax highlighting it supports graphical construction of Pig Latin scripts (box and arrow) as well as showing the user the output schema of each line in a Pig Latin script and example output of his or her script. Pig can be run directly from PigPen in either local or Map-Reduce mode. Has not been maintained for a while.
Emacs Pig Latin Mode
...
pig-mode is an Emacs major mode for editing Pig scripts. It supports syntax highlighting and indentation for Pig versions 0.2 to 0.11 and is actively maintained.
http://codegithub.google.com/p/pigpy/cloudera/piglatin-mode
A simple Emacs mode that highlights Pig Latin syntax. It's very basic, but it does make life more pleasant if you are an Emacs user. Works for Emacs 23.
Gedit
Syntax highlighting for Pig Latin. Versions exist for Pig 0.11 - 0.15. Included in gedit as of Gnome 3.16.
https://github.com/eyala/gedit-pig
Pig Editor (web-based)
This web-based Pig Editor based on Hue lets you edit your Pig script with auto completion and syntax highlighting. You can save your scripts and execute them with a single click, directly from your browser. Logs and results are displayed within this one-page app for a better interactivityA Python wrapper that helps users manage their Pig processes. It can manage many similar Pig Latin scripts, including running common root scripts and caching the results to be used in generation of the final output scripts. Pig can be run directly from PigPy, allowing users to inspect results of the Pig job and take further actions.
TextMate Plugin
http://www.github.com/kevinweil/pig.tmbundle
A plugin for the Mac TextMate editor.
Vim Plugin
http://www.vim.org/scripts/script.php?script_id=2186
Provides syntax highlighting for Pig Latin scripts (up to version 0.12).
Snipmate plugin (Snipmate is a Vim plugin for code completion)
https://github.com/rverk/snipmate-pig
Intellij
https://github.com/brandonkearby/intellij-pig
CodeMirror 2 Pig Latin Mode
Demo site: http://codemirror.net/mode/pig/index.html
Github project site: http://prasanthj.github.com/pig-codemirror-2/
Javascript based online editor with syntax highlighting for Pig Latin.
Sublime editor plugin
https://github.com/matthayes/sublime-text-pig
Notepad++
Syntax highlighting and auto complete for Pig 0.15.