Apache Kafka on Windows

A high-throughput distributed messaging system.

Short Introduction :

  • Originally created by LinkedIn, contributed to Apache in July 2011.
  • Kafka is written in Scala.
  • Multi-language support for Publish/Consumer API.
  • Active deployment, 0.8 out now.

Install Kafka on Windows:

NOTE: You need to install Java SE Server JRE and update the system environment path variable like here.

I downloaded the 0.8.2.1 version under Binary downloads Scala 2.10 – direct link.

After unzipping the .tgz files with 7zip, the folder structure will look like below snapshot. I placed it in E:/Kafka.

Downloaded_Unzipped_Kafka

You can also check there will be windows folder inside bin folder.

Now we also have to install Zookeeper for Kafka as Kafka uses Zookeeper for storing variety configuration values. You can find more information here.

Install Zookeeper: I downloaded Zookeeper – 3.5.0 –alpha from downloadsdirect link.

After unzipping .tar.gz file using 7zip, the folder structure will look like below. I placed it in E:/Zookeeper.

Downloaded_Unzipped_Zookeeper

Open zoo.cfg file placed in “Zookeeper-3.5.0-alpha/conf” and Change the dataDir path in zoo.cfg file. I set the path to E:\Zookeeperdata.

image

Now, First we will start Zookeeper and then Kafka.

Right Click zkServer.cmd file and click Run as administrator. It will be inside “Zookeeper-3.5.0-alpha/bin”.

Start_Zookeeper

Hurray! Zookeeper started successfully.

image

Now Open server.properties file placed in “Kafka_2.10-0.8.2.1/config”. Change the log.dirs path to some valid path. I changed it to E:/Kafka/kafka-logs.

image

Open command window and run the kafka-server-start.bat and pass server.properties file path like below snapshot.

image

Hurray! Kafka is running.

image

Advertisements

Learning Nodejs for building fast responsive WebSites – Part1.

Few days earlier just surfing heard about nodejs and its rapid fast development and growing popularity.

Node.js is an InfoWorld 2012 Technology of the Year Award Winner.

What is nodejs :
Ya you heard right, Nodejs is server side javascript. Before Nodejs we run the javascript inside the browsers only at client side. But Nodejs makes the javascript run outside the browser, by making use of Google’s V8 VM, the same runtime environment for JavaScript that Google Chrome (browser) uses.

Getting Started:

  1. Install Nodejs on your machine. (mine – windows)
    nodejs windowsinstaller
    Installing it, will install node.exe, npm (node packet manager), and some modules in Program Files folder.
  2. Now open cmd (command prompt) and type
    >node -v // to check the correct installation and it will tell you the version of nodejs installed.
  3. Create new folder with any name. I created node_learning.
    Create a file Hello.js and inside it write console.log("Hello World");
    Thats it save it.
    And run from cmd now.
    >cd node-learning // to go to that file's path
    node_learning>node Hello.js

Create a Web Page with server :

  1. Creating a Server –
  2. Create a file server.js and write code
    var http = require("http"); //instantiate the http module from nodejs module
    http.createServer(function(request, response) {
    response.writeHead(200, {"Content-Type": "text/plain"});
    response.write("Hello World");
    response.end();
    }).listen(8888); //port number to listen

    Save this file.

  3. Run it –
  4. Open cmd. Type
    >node server.js
    Nothing will happen only server will start.
    Now Open browser (like mozilla or chrome)
    Open page/url localhost:8888 and the response Hello World will get printed there.
    U can pass html also in place of Hello World text in server.js file
    Thats all. Now you have to explore further.
    For exiting node server in cmd press ctrl + c

Benefits & Conclusion :
There are huge benefits of nodejs. It is very fast and responsive.
Very good for small portfolios, websites and blogs.

Here you have successfully installed nodejs. Build a Hello World static web page.

What Next
In the next part, I will get u make a small website using nodejs with mvvm pattern using express js. Also use of npm would be their

Posting Image directly in Twitter using Status with Media in C#

I got this task. And after searching a lot, i acheived this.

I was using TweetSharp api in my application.So i tried to achieve with that.
It gives you solution so that you can achieve that by Twitpic Api.

Then I read POST statuses/update_with_media

Tried a lot, not able to do with the C# twitter api’s too like TweetSharp, Hommock etc..
Then Here is the solution 🙂

Step 1: Make Oauth Authorization Header

Simple, Download OAuth.Manager dll from here

Add Reference in your project. And add namespace (Using OAuth;) in your page.

Note : I am assuming here you have made the request to the user before and have configuration key, configuration secret, access token and access secret. You can do it by any C# api.

Step 2: Using this dll and generating header

Setting request Values :

var oauth = new OAuth.Manager();
oauth["consumer_key"] = ~Your application's Key~;
oauth["consumer_secret"] = ~Your application's Secret~;
oauth["token"] = ~Access token~;
oauth["token_secret"] = ~Access secret~;

Making Request :

var url = "https://upload.twitter.com/1/statuses/update_with_media.xml";
var authzHeader = oauth.GenerateAuthzHeader(url, "POST");
var request = (HttpWebRequest)WebRequest.Create(url);

request.Method = "POST";
request.PreAuthenticate = true;
request.AllowWriteStreamBuffering = true;
request.Headers.Add("Authorization", authzHeader);

Step 3: Adding image to OAuth request and Posting

Here is the message to be uploaded sent as status
string message = "Hello Image Demo";

Here is the image
string imageFile = HttpContext.Current.Server.MapPath("/Images/abc.png"); //Sample image

if (imageFile != null)
{
string boundary = “~~~~~~” +
Guid.NewGuid().ToString().Substring(18).Replace(“-“, “”) +
“~~~~~~”;

var separator = “–” + boundary;
var footer = “\r\n” + separator + “–\r\n”;
string shortFileName = Path.GetFileName(imageFile);
string fileContentType = “image/png”;
string fileHeader = string.Format(“Content-Disposition: file; ” +
“name=\”media\”; filename=\”{0}\””,
shortFileName);
var encoding = System.Text.Encoding.GetEncoding(“iso-8859-1”);

var contents = new System.Text.StringBuilder();
contents.AppendLine(separator);
contents.AppendLine(“Content-Disposition: form-data; name=\”status\””);
contents.AppendLine();
contents.AppendLine(message);
contents.AppendLine(separator);
contents.AppendLine(fileHeader);
contents.AppendLine(string.Format(“Content-Type: {0}”, fileContentType));
contents.AppendLine();

// actually send the request
request.ServicePoint.Expect100Continue = false;
request.ContentType = “multipart/form-data; boundary=” + boundary;

using (var s = request.GetRequestStream())
{
byte[] bytes = encoding.GetBytes(contents.ToString());
s.Write(bytes, 0, bytes.Length);
bytes = File.ReadAllBytes(imageFile);
s.Write(bytes, 0, bytes.Length);
bytes = encoding.GetBytes(footer);
s.Write(bytes, 0, bytes.Length);
}
}

Now Request Made Getting response

using (var response = (HttpWebResponse)request.GetResponse())
{
if (response.StatusCode == HttpStatusCode.OK)
{
//Posting Successfull
}
}

All Done. Image Posted Successfully 🙂

Problem Solved : After Downloading file, Update progress remains

What the Problem was ? I made an update panel, inside it was a file link(link button). On click of the file link update progress starts showing the loading image, when the download box comes either you save, open or close it. Loading/Spinning image is still there.

Actual code creating the problem was ? Response.redirect was redirecting the page to FileDownload.aspx ,from where the download box(file) comes with the save, open or close options.

Solution : Remove the Response.redirect(“FileDownload.aspx”) and redirect to this page with javascript.

STEP 1: Replace the Response.Redirect(“FileDownload.aspx”) with this :

ScriptManager.RegisterClientScriptBlock(this, this.GetType(), "Download", "GotoDownloadPage();", true);

this will call the javascript function GotoDownloadPage();

STEP 2 : Define the function within javascript script tag as

GotoDownloadPage()
{
window.location=”FileDownload.aspx”
}

Problem Solved.

Tip: if you want to send some documentId like parameters to the FileDownload page, you can use session variables.

Thnx. Happy Coding…

Problem Solved : FileUpload control inside GridView

How the problem arises? I made a gridview inside it was a fileupload control, in a column on every row with Update(Link Button). When the upload button is clicked, the file gets uploaded along the row. Now to show the Uploaded file , the aspx page needs to be reload and this caused the problem.

What the problem was? If you are in some inner page of the gridview like say fifth and you uploaded the file in certain row there after uploading the file you are being redirected to the first page of the gridview.

Some Initials You may know to solve the problem : I was having general paging in the gridview.
GridView1_PageIndexChanging()
{
GridView1.PageIndex = e.NewPageIndex;
GridView1.DataBind();
}

I was handling the Upload click with command=”Upload”
and after handling it, i made a Response.Redirect("samepage.aspx");

Solution : I maintained the paging for this problem by taking two session variables, one for storing the page index in session variable every time and another one to ensure the Upload button is clicked.

I did this in 3 easy steps:
Step 1 :
Add the following line :
GridView1_PageIndexChanging()
{
GridView1.PageIndex = e.NewPageIndex;
//Making session variable to know the page index
Session["PageIndex"] = e.NewPageIndex;
GridView1.DataBind();
}

Now i know the page index, everytime the page index got clicked.

Step 2 :
Making the page to know the update button is clicked
Add the following line above redirecting the page to self :
Session["PageUploaded"]="true";
Response.Redirect("samepage.aspx");

Step 3 :
Now I have to just handle the response.redirect of Upload Button Click.

protected void Page_Load(object sender, EventArgs e)
{
if (!Page.IsPostBack)
{
//place this code above binding data to GridView
if(Session["PageUploaded"] != null)
{
GridView1.PageIndex = int.Parse(Session["PageIndex"].ToString());
Session["PageUploaded"] = null;
}
GridView1.DataBind();
}
}

Thnx. Happy Coding…