Sarav's Weblog

Technical Articles for RoR Developers

Category Archives: Ruby

Apache Thrift with Ruby – A basic Tutorial

Thrift is a software framework for scalable cross-language services development. It combines a software stack with a code generation engine to build services that work efficiently and seamlessly between C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, Smalltalk, and OCaml.

Originally developed at Facebook, Thrift was open sourced in April 2007 and entered the Apache Incubator in May, 2008.

Thrift is arguably one of the best lightweight systems for cross-language programming using code-generation, RPC, and object serialization. Designed from the ground up by the Facebook development team it enables seamless integration of the most commonly used languages (Ruby, Perl, C/C++, Haskell, Python, Java, and more) via a set of common RPC libraries and interfaces.

In this post I’ll outline how to setup and connect a thrift client in ruby

To install thrift

gem install thrift

Steps:

  • Defining a thrift service
  • Run the server script(ruby)
  • Run the client script(ruby)

Defining a service in Thrift

The core goal of Thrift is to enable efficient and reliable communications across multiple languages. All datatypes and services are defined in a single language-neutral file and the necessary code is auto-generated for the developer by the thrift compiler.

One of the great things about thrift is that it simplifies the communication between a client and server to a simple .thrift file. This file describes the data structures, and functions available to your remote service. In this tutorial, we use a simple calculator mechanism.

Define a struct and a service

	enum BinaryOperation {
		ADDITION = 1,
		SUBTRACTION = 2,
		MULTIPLICATION = 3,
		DIVISION = 4,
		MODULUS = 5
	}

	/** Structs are the basic complex data structures. They are comprised of fields 
	   * which each have an integer identifier, a type, a symbolic name, and an 
	   * optional default value. */
   
	struct ArithmeticOperation {
		1:BinaryOperation op,
		2:double lh_term,
		3:double rh_term,
	}

	/* Structs can also be exceptions, if they are nasty. */
	exception ArithmeticException {
		1:string msg,
		2:optional double x,
	}


	service Calculator {

		/**
		* A method definition looks like C code. It has a return type, arguments,
		* and optionally a list of exceptions that it may throw. Note that argument
		* lists and exception lists are specified using the exact same syntax as
		* field lists in struct or exception definitions.
		*/
		
		double calc(1:ArithmeticOperation op) throws (1:ArithmeticException ae),
		
		
		/**
		* This method has an oneway modifier. That means the client only makes
		* a request and does not listen for any response at all. Oneway methods
		* must be void.
		*
		* The server may execute async invocations of the same client in parallel/
		* out of order.
		*/
		
		oneway void run_task()
	}

Open a empty file and create service(copy the above code) and save it with .thrift extension(eg:calc.thrift)

We’ve defined two different functions: remote calculationand an asynchronous method call. To generate the required code, we simply call the thrift generator:

# Generate C++, Ruby, and Python implementations
# generated code will be in ‘gen-cpp’, ‘gen-rb’, ‘gen-py’ folder

thrift --gen rb calc.thrift

The beauty of this approach is, of course, the ability to mix and match implementations of services: server may be written in C++, but we can access it seamlessly via a Python, Java, or a Ruby client. Thrift will take care of the communications links, object serialization, and socket management!

Implementing a Thrift powered server in Ruby

A very simple example of interaction with a calculator server application whose actions are facilitated by thrift. Both the client and server negotiate on the common interface defined by calc.thrift.

# include thrift-generated code
$:.push('../gen-rb')

require 'thrift'
require 'calculator'

# provide an implementation of Calculator
class CalculatorHandler
	def initialize()
		#nothing goes here
	end

	def calc(val)
		lh_term = val.lh_term
		rh_term = val.rh_term
		case val.op
			when 1 #Addition
				lh_term+rh_term
			when 2 #subraction
				lh_term-rh_term
			when 3 #multiplication
				lh_term*rh_term
			when 4 #division
				lh_term/rh_term
			when 5	#modulas
				lh_term%rh_term
		end
	end
	
	def run_task()
		puts "Kicking off the task"
	end
end

# Thrift provides mutiple communication endpoints
#  - Here we will expose our service via a TCP socket
#  - Web-service will run as a single thread, on port 9090

handler = CalculatorHandler.new()
processor = Calculator::Processor.new(handler)
transport = Thrift::ServerSocket.new(9090)
transportFactory = Thrift::BufferedTransportFactory.new()
server = Thrift::SimpleServer.new(processor, transport, transportFactory)
puts "Starting the Calculator server..."
server.serve()
puts "done."

To expose our thrift service, we wrap it into a TCP socket listening on port 9090. From this point on, Thrift takes over all communications, serialization and handling of the incoming requests. Because the protocol is identical in every language, the client may be written in any language of choice.

Building the Ruby Client

You may have the client program in any language which supports Thrift.

In similar fashion, we can build a Ruby client for any Thrift service with just a few lines of code. To interface with our server implementation above, refer the following code:

# include thrift-generated code
$:.push('../gen-rb')

require 'thrift'
require 'calculator'

begin
	port = ARGV[0] || 9090

	transport = Thrift::BufferedTransport.new(Thrift::Socket.new('localhost', 9090))
	protocol = Thrift::BinaryProtocol.new(transport)
	client = Calculator::Client.new(protocol)

	transport.open()

	ar = ArithmeticOperation.new()    
	ar.op = BinaryOperation::ADDITION
	ar.lh_term = 99
	ar.rh_term = 3
	
	# Run a remote calculation
	result = client.calc(ar)  #it accessing the ruby server program method calc via thrift service
	puts result.inspect
	
	#Run a Async call
	client.run_task()
	
	transport.close()
rescue
	puts $!
end

And there you have it, a simple thrift client. I hope you think about thrift next time you need to interface between 2 applications, or require some kind of client->server model for your system. Thrift makes it very fast and convenient to share data across different programming languages and makes it easy/flexible to develop to a shared interface contract. Cheers!

Download Files
calc.thrift
ruby_server
ruby_client

Summary:

  • Download the calc.thrift and place it into the new folder(name it whatever you like. Eg:- thrift-ruby)
  • cd thrift-ruby
  • thrift –gen rb calc.thrift (it will create the auto generated ruby code. Dont change anything on the auto generated code)
  • create the folder named “rb” and place the ruby_server.rb and ruby_client.rb
  • run server program first. $ruby ruby_server.rb (it will start the server)
  • then run the client program in new cmd prompt. $ruby ruby_client.rb

References:

Advertisements

Ruby-LDAP (add, search)

LDAP in Ruby

To use LDAP in Ruby you have to use following gems. So Install the following gems.

  • sudo gem install net-ldap
  • sudo gem insall ruby-net-ldap

User Authentication against LDAP Directory Using Ruby

Example of a user-authentication against an LDAP directory:

require 'rubygems'
require 'net/ldap'

ldap = Net::LDAP.new
ldap.host = your_server_ip_address
ldap.port = 389
ldap.auth "joe_user", "opensesame"
if ldap.bind
  # authentication succeeded
else
  # authentication failed
end

Search against LDAP Directory Using Ruby

Quick Example of a search against an LDAP directory:

require 'rubygems'
require 'net/ldap'

username = "cn=manager,dc=example,dc=com"
password = "xxxxxxxx"
host = example.com #(or) host = 111.111.111.11 (ipaddress)
ldap = Net::LDAP.new :host => host,  :port => 389
ldap.auth username, password

filter = Net::LDAP::Filter.eq( "cn", "George*" )
treebase = "dc=example,dc=com"

ldap.search( :base => treebase, :filter => filter ) do |entry|
  puts "DN: #{entry.dn}"
  entry.each do |attribute, values|
    puts "   #{attribute}:"
    values.each do |value|
      puts "      --->#{value}"
    end
  end
end

#puts  ldap.get_operation_result

Check out the rest of the documentation for pretty good examples. This is the library I recommend. In my situation, I’m using ruby-net-ldap to import data in to, manipulate and query data in an OpenDS LDAP server.

Making beautiful graphs with Ruby

I’m evaluating some charting tools (free) to be used with RoR web
application. Those should be easy to use and yet have some elegant
features. At the moment I’m using Open Flash chart. But I’m on
evaluating following

Curb Gem – Libcurl bindings for Ruby

Curb – Libcurl bindings for Ruby

  • * curb.rubyforge.org/
  • * rubyforge.org/projects/curb

Curb (probably CUrl-RuBy or something) provides Ruby-language bindings for the libcurl(3), a fully-featured client-side URL transfer library. cURL and libcurl live at curl.haxx.se/ .

Curb is a work-in-progress, and currently only supports libcurl’s ‘easy’ mode.
License

Curb is copyright (c)2006 Ross Bamford, and released under the terms of the Ruby license. See the LICENSE file for the gory details.
You will need

* A working Ruby installation (1.8+, tested with 1.8.5)
* A working (lib)curl installation, with development stuff (7.5+, tested with 7.15)
* A sane build environment

Installation…
… will usually be as simple as:
$ gem install curb

Or, if you downloaded the archive:
$ rake install

If you have a wierd setup, you might need extconf options. In this case, pass them like so:

$ rake install EXTCONF_OPTS='--with-curl-dir=/path/to/libcurl --prefix=/what/ever'

Currently, Curb is tested only on GNU/Linux x86 – YMMV on other platforms. If you do use another platform and experience problems, or if you can expand on the above instructions, please get in touch via the mailing list on Curb’s Rubyforge page.

Curb has fairly extensive RDoc comments in the source. You can build the documentation with:

$ rake doc

Examples
Simple fetch via HTTP:

c = Curl::Easy.perform("http://www.google.co.uk")
puts c.body_str

Same thing, more manual:

c = Curl::Easy.new("http://www.google.co.uk")
c.perform
puts c.body_str

Additional config:

Curl::Easy.perform("http://www.google.co.uk") do |curl|
curl.headers["User-Agent"] = "myapp-0.0"
curl.verbose = true
end

Same thing, more manual:

c = Curl::Easy.new("http://www.google.co.uk") do |curl|
curl.headers["User-Agent"] = "myapp-0.0"
curl.verbose = true
end

c.perform

Supplying custom handlers:

c = Curl::Easy.new("http://www.google.co.uk")

c.on_body { |data| print(data) }
c.on_header { |data| print(data) }

c.perform

Reusing Curls:

c = Curl::Easy.new

["http://www.google.co.uk", "http://www.ruby-lang.org/"].map do |url|
c.url = url
c.perform
c.body_str
end


HTTP POST form:

c = Curl::Easy.http_post("http://my.rails.box/thing/create",
Curl::PostField.content('thing[name]', 'box',
Curl::PostField.content('thing[type]', 'storage')


HTTP POST file upload:

c = Curl::Easy.new("http://my.rails.box/files/upload")
c.multipart_form_post = true
c.http_post(Curl::PostField.file('myfile.rb'))

Resource from: http://curb.rubyforge.org/

How to use helpers in ruby Console?

We can use rails helper methods in ruby console by using “extend” method.

Example:

>> helper.extend ApplicationHelper
>> helper.number_to_currency(232324)
=> "$232,324.00"

blackbook gem (Contact Importer gem for Ruby on Rails)

Blackbook automates the nitty-gritty of importing contacts from various services and files and exporting them as VCard, XML, or simple Hash. Utilize those contacts from services like AOL, GMail, Yahoo Mail, Hotmail or CSV to help your social networking site become GIGANTIC overnight! You‘ll be able to get big and sell for millions before anyone figures out it‘s just like every other social network.

Websitehttp://contentfree.rubyforge.org/blackbook/

FEATURES/PROBLEMS:

The current list of supported services and file types:

Import:

  • AOL
  • CSV files
  • Gmail
  • Hotmail
  • Yahoo! Mail

Export:

  • Simple hash (default)
  • Vcard
  • XML

If you create an additional importer or exporter – or simply find a bug – please consider submitting it as a patch to the project so the community can all benefit from your hard work and ingenuity.

SYNOPSIS:

# An example of fetching Gmail contacts – by default, returns an array of hashes with :name and :email

  contacts = Blackbook.get :username => 'me@gmail.com', :password => 'whatever'

# or returning XML

  contacts = Blackbook.get :username => 'me@gmail.com', :password => 'whatever', :as =>  :x ml

# or importing from a CSV file

  contacts = Blackbook.get :csv, :file => #<File:/path/to/file.csv>

REQUIREMENTS:

  • Mechanize and its dependencies, for interacting with online providers
  • Fastercsv for reading CSV, Mechanize >= 0.7.0 for page scraping

INSTALL:

  • sudo gem install blackbook


Paypal on Rails – ActiveMerchant tips

Anyone who has looked into using ActiveMerchant and PayPal together knows it is quite a challenge. ActiveMerchant is great, but there is no documentation for using it with Paypal. Here are some tips to help you get it working:

Here is a page with some good references.

Setting up Paypal

The paypal website is a bit tough to navigate and they don’t make it obvious to figure out what are all the things you need to do to get it all setup.

  1. You will need a sandbox (developer) paypal account to test your set up. Sign up here
  2. The sandbox does not send emails, so you will have to use the Sandbox Email Tab to validate your sandbox account email address.
  3. Launch the sandbox
  4. Create a sandbox business account. and go through the normal paypal verification stuff. (It does not use your real bank account.)
  5. Go to the Merchant Services tab and signup for a Website Payments Pro Account.
  6. Verify your merchant account. You will be asked to enter you SSN number, use 111 + six random numbers. If you get an error message, try a different six numbers, the fake ssn must be unique in the sand box.
  7. Under the My Account tab and the Profile subtab and the Billing Agreements link, make sure you accept the Agreement.
  8. Same for the PayPal Monthly Billing link.
  9. Follow these instructions to create your API Certificate. Do not make the API Signature. You only need to do the “Step 1. Generate Certificate” part.
  10. Make sure you save the username & password given and download and save the pem file.

Paypal should now be set up. You will have to go through and do all the same stuff for your real paypal account, once your development is done and working.

Setting up ActiveMerchant

  1. Install ActiveMerchant by following the instructions, you will need to install the money gem, which ActiveMerchant requires.
  2. Rename the pem file to cert_key_pem_dev.txt and put it in your config dir.
  3. Add the following code to your environments/development.rb:
  4. 
    config.after_initialize do
      ActiveMerchant::Billing::Base.gateway_mode = :test
      ActiveMerchant::Billing::PaypalGateway.pem_file = File.read(RAILS_ROOT + '/config/cert_key_pem_dev.txt')
    end
    $PAYPAL_LOGIN = ''
    $PAYPAL_PASSWORD = ''
    
  5. Use the actual username and password in the previous code (don’t use the empty strings 🙂
  6. Do the same for the environments/test.rb
  7. Do the same for the environments/production.rb, except remove the line: ActiveMerchant::Billing::Base.gateway_mode = :test and use the cert_key_pem_prod.txt file (which you can get through your real paypal account (renaming to cert_key_pem_prod.txt ).
  8. Do something like this in your code:
  9. 
    
    amount = 1000 #$10.00
    credit_card = ActiveMerchant::Billing::CreditCard.new(
        :type       => 'visa',
        :number     => '4242424242424242',
        :month      => 8,
        :year       => 2009,
        :first_name => 'Bob',
        :last_name  => 'Bobsen',
        :verification_value=> '123'
    )
    
    flash[:error] = credit_card.errors and return unless credit_card.valid?
    
    billing_address = {
        :name     => "John Smith",
        :address1 => '123 First St.',
        :address2 => '',
        :city     => 'Los Angeles',
        :state    => 'CA',
        :country  => 'US',
        :zip      => '90068',
        :phone    => '310-555-1234'
    }
    
    gateway = ActiveMerchant::Billing::PaypalGateway.new(:login=>$PAYPAL_LOGIN, :password=>$PAYPAL_PASSWORD)
    
    res = gateway.authorize(amount, credit_card, :ip=>request.remote_ip, :billing_address=>billing_address)
    
    if res.success?
        gateway.capture(amount, res.authorization)
        flash[:notice] = "Authorized"
        redirect_to somewhere_url
    else
        flash[:error] = "Failure: " + res.message.to_s
    end
    
  10. Make sure State and Country are two character code.
  11. The credit card errors is actually a hash, not an array (like AR.errors), so you will need a helper to display the error.

Last Tip

  1. When testing with paypal, you must use port 80: ./script/server -p 80