Atom Login Admin

Seeds of mention

Google App Engine Pipeline API

lakshmi@cloudysunny14.org
mentioned on Saturday,January 05,2013

Web Crawler of Lakshmi Projects that I have made ​​is using Google App Engine Pipeline API.
What is Pipeline API? There is Framework that introducing in project page;
which can be connected together to operate as Cascade,It also allows parallel processing.
Introduce simple way to run using unittest that according to sample code:

  • test.py
import unittest
import pipeline

class DivideWithRemainder(pipeline.Pipeline):

  output_names = ['remainder']

  def run(self, dividend, divisor):
    self.fill(self.outputs.remainder, dividend % divisor)
    return dividend // divisor

class EuclidGCD(pipeline.Pipeline):

  output_names = ['gcd']

  def run(self, a, b):
    a, b = max(a, b), min(a, b)
    if b == 0:
      self.fill(self.outputs.gcd, a)
      return
    result = yield DivideWithRemainder(a, b)
    recurse = yield EuclidGCD(b, result.remainder)

class GCDTest(unittest.TestCase):
  def setUp(self):
    """Initialize temporary application variable."""
    unittest.TestCase.setUp(self)
    pipeline.testutil.setup_for_testing()

  def testGCD(self):
    """Test for GCD Pipeline"""
    stage = EuclidGCD(1071, 462)
    stage.start_test()
    self.assertEqual(21, stage.outputs.gcd.value)

if __name__ == "__main__":
  unittest.main()
  • Run&Result
.
----------------------------------------------------------------------
Ran 1 test in 0.276s

OK

It is useful for concurrent jobs, Also support for Asynchronous operations as Waiting for user operations. which can to be more efficiency.
Google App Engine MapReduce is created by this framework.
Next, I will introduce Google App Engine MapReduce. coming soon:).

Comments

Add Comment

Login
This entry was tagged #GAE