Archive

Posts Tagged ‘Cloudbees’

Connecting to an existing Cloudbees repository from Windows

December 29, 2012 Leave a comment

This post is the result of trying to figure out how to setup a repository clone of one of my projects on my windows machine.  I work on my Mac most of the time, but at times it’s more convenient to work on my Win machine (usually when I have to debug IE issues, and need to try js fixes to see what impact they make)

So, lets start by setting up our tools and IDE on Windows.  We obviously need Java, Eclipse, and Maven, and actually that’s it.  In Eclipse, install m2e and EGit plugins.  Make sure you follow all the installation instructions for Maven with all the variables in the right place.

Now we need to setup SSH.  This needs to happen in two places, in Eclipse settings and in Cloudbees settings.  Here are the steps:

  1. In Eclipse, go to preferences->General->Network Connections->SSH2, select key management, and generate a DSA key.  Enter the pass phrase and remember it, you’ll need it in a minute.  Copy the public key to the clipboard.
  2. Click “Save private key” and save it, go to general and add the saved private key.   Apply and save.
  3. Go into Cloudbees settings->SSH keys, choose a name and paste the public key from the clipboard.  Click “Add”.
  4. Go to Cloudbees->Repositories, go into your project and copy to clipboard the SSH url unique to your repository.
  5. Go back to Eclipse, File->import, choose Git->Projects from Git.  Choose URI and paste the SSH URL you copied from your repository.  Choose ssh as the protocol.  The user should be git, and password empty.  Port can remain empty (it will be filled with the default which is 22)
  6. You will now need to enter your pass phrase.
  7. Next next next and your project will be cloned locally.
  8. To help maven work you need to go into Eclipse->Preferences->Maven, and go to installation->Add and pick the Maven installation you installed locally (instead of the Embedded one).
  9. If you run into problems with your project when you open the IDE, see this post.  If you still have problems, and you had Maven installed before, try deleting the .m2 resources directory.  You might also need to go into Preferences->Maven->User settings and reindex or update settings.
  10. Your project should automatically compile.

Problems you may encounter:

1. M2E installation fails and maven does not seem integrated into eclipse.  I couldn’t fix that and had to reinstall eclipse

2. You get complaints about javac and JRE: you’re using JRE instead of JDK, switch to JDK in eclipse->java->installed jdk’s

Advertisements
Categories: R&D Tags: , , , , , , ,

Creating an online service – Using cloudbees for Continuous Deployment with Basic Service based projects

November 5, 2012 2 comments

The previous post discussed getting the basic service application to your local development environment and getting it setup.  In this post I’ll describe the process of setting up cloudbees for working with a Basic Service like project, so we can close the loop all the way to production.

We’ll be using cloudbees’ SEND-GRID services to send emails (registration verification process uses it to send a confirmation email), so before going further, go into cloudbees->services, and activate SendGrid.  You can pick the free account for now.

Before creating a new git repository in cloudbees, we need to do some housekeeping in Git.  Git has a file called .gitignore, which helps tell git what you don’t want to monitor and commit.  At the very least, add the following entry:

/target

to the .gitignore file (create the file if it does not exist in your main project directory).  This will prevent you from accidentally checking in your targe directory (which is built automatically by the cloudbees CI environment).  Now that we have this covered, it’s time to get started.

Setting up a git repository in cloudbees is easy, just pick Repositories->create new repository, give it a name and pick Git as your repository type, and click enter.

Once that repository is done, you need to connect your local repository to the cloudbees one so you can push your code.  After creating your repository, you will see some helpful short examples of how to connect to cloudbees (just below the details paragraph).  Pick “Migrating to your cloudbees repository”, and follow the steps.  Don’t pull/push from legacy, just use the remote rm command to remove the old repository.  Alternatively, you can delete the .git directory and start a new git repository from scratch (without the basicservice history).  Just follow the “committing an existing project to git” cloudbees instructions to do that instead.

You repository should now be about 0.6MB, and contain the project files.

Once you’ve setup your repository, the next thing you need to do is setup a Build configuration.

Before going to builds, open your repository details, and copy the uri starting with ssh:// (you will need it shortly)

Hit builds, and create a New Job.

Give it a name, and pick “Build a maven 2/3 project”.  Hit next.

Mark “one click deployment”, select “Oracle 1.7 latest” as your JDK, unclick “restrict where this project can run”, select “git” as your repository.

In the git url paste the ssh:// url you copied earlier.  Put two stars (**) in the branches to build input field.

In build->goals and options put: clean install

In post steps, select “run only if build succeeds”, in the build settings, mark the email notification and add your email so you get notified of checkins and builds.

Pick “abort the build if it’s stuck” and pick whatever makes sense to you (I chose elastic)

Add a post-build-action and deploy-to-cloudbees, add web application and pick “maven artifact selection”.  Add an application id (your project name is fine).  Click “Advanced”, and enter production in the “Environment” field.

Don’t worry about the “not found” warning, click save.

Click build Now, and when an available worker is ready your project will start building. Click the running build and then console Output to see the process in action.  When the build is complete, you should see something like:

[cloudbees-deployer] Deployed to application id your_account/your_project
[cloudbees-deployer] Can be accessed at http://yourproject.youraccount.cloudbees.net

Click it and start playing around with your production online service!

One last small thing left to do: setup our domain.  We’ll cover this in the next post.

Categories: R&D Tags: , , , ,

Online service development in java – a beginner’s guide

November 4, 2012 1 comment

In the coming series of posts, I will share my development experience (code included) with creating an online service from scratch.  If you read through all posts, you should reach a point where you are capable of writing code and deploying it to a production like environment which is available online, using some of the latest (and greatest) open source libraries available in java.  You will definitely not be at the end of the journey, but will get a pretty good head start.

I’ve created a sample project, which basically does nothing (well, almost nothing), but it does so in a (semi) secure way, using several top libraries.  Basic service, is a service which lets you register, login, and… do nothing.  But, it does so using:

Spring, Maven, ESAPI (+AppSensor) for input validation, JSP’s (JSTL+Tiles 2), jQuery (+UI), Mongo DB, Logback, Jackson, Mockito (+PowerMock) for unit testing, and it does it in a RESTful way, using complete Data and Presentation separation (all JSP’s get their data through REST requests, meaning you can add easily add mobile support without almost any change to the  interface), and is fully internationalized (currently supports hebrew and english, meaning we have RTL languages covered).  The complete source code for Basic service is available on google code, it is a fully functional (and GPL open source) example of combining all of these technologies , which means that you can concentrate on the next steps rather than spend time building the foundation from scratch.  It took me about 3 months to go through all the baby steps in each of those technologies (after spending some time picking the right ones of course), I hope this series will save you at least some of that time.

Please keep in mind that I do not claim this to be a complete and bullet proof piece of software, there is still much to do in order to make it “production ready”.

Basic service is deployed on Cloudbees, using Continuous Deployment,  you’ll get some insights on how to do that as well.

One final note: the title says “a beginner’s guide”, but should be read with two different interpretations: one is (possibly) you, who is just getting started with creating an online service, and the other is me, sharing my experience with jumping into these deep water a couple of months ago.  If you like what you read, you’re welcome to comment, if you see me making horrible mistakes, constructive criticism is more than welcome.

Let’s move on.  The next post will cover the basics of setting up a functional development environment, one which will allow you to develop and instantly deploy your code to production so it is available to the rest of the world

Copying a git repository from Cloudbees to Github

October 16, 2012 Leave a comment

I recently had to take my cloudbees hosted repository hosted on, well, cloudbees, and move it to github.  This was a bit tricky for someone will little knowledge of ssh and git (me), so hopefully if you’re in my situation this will help.  Note that this was done on a Mac OSX, should be similar for other unix based OS’s but probably slightly different for Windows.

I’ve used green color to represent terminal commands, so if you see something in green, it obviously needs to go into terminal…

First create a new account and repository on github.

Once you do that, you need to setup your ssh connection to git (you can also connect using http, but I’ll stick to SSH for now).  There’s a pretty good explanation by git-hub here, which I followed (well, some of it), but basically here’s what you need to do:

  1. Go to command terminal, and enter the following command which will copy your ssh public key to the clipboard
    pbcopy < ~/.ssh/id_rsa.pub
  2. If id_rsa.pub was not found, you need to generate a new ssh key locally, follow the git-hub tutorial mentioned above to do that.
  3. Go to your github account settings->ssh keys, and click “add SSH key”.  Give it a name and paste your key.
  4. In terminal, enter
    ssh-add -l
  5. This should give you a print of your key hash, make sure it is identical to what you see in your git-hub listing.  It should look like this:
    1a:12:22:f8:61:a4:48:b9:d2:92:aa:02:74:82:a2:d7
  6. In terminal, go to your repository directory (e.g. ~/code/basicservice/)
  7. make sure you have a git repository there:
    git status
  8. Create a new repository on git-hub, don’t initialize it with README
  9. Push your repository to git-hub
    git push git@github.com:<youraccount>/<your-repository>.git master
    you can get the git url from your git repository page, make sure you click on SSH to get the right url, or just replace <youraccount>/<your-repository> with your data.
  10. That’s it.  Your repository is now hosted on git-hub as well, you can now drop the cloudbees repo and start using git-hub.  To make it push both to cloudbees and to github simultaneously (if you want to keep both) you need to add some configuration, and I’ll pass on that right now.  You can check out this post for help on that.
Categories: R&D Tags: , , , ,

Using REST in cloudbees – not so simple

September 13, 2012 2 comments

Well, this one is going into “things that can drive you mad” for sure, as it took me a couple of days to reach the inevitable conclusion that I have to give up 😦

So here’s the thing:  you created a RESTful application, using the proper GET/PUT/POST/DELETE headers where appropriate, it works great on your local Jetty/Tomcat deployment, but when you try to run it on the cloud, it fails.  After much testing and debug prints, you find that when using PUT & DELETE the request gets there BUT the body embedded  parameters do not.

If this sounds familiar, you’ve got a problem.  As it is right now (Sep. 2012), app servers are not obligated to parse parameters delivered with PUT/DELETE requests if they are sent in the body (like in POST), because the standards to do not define that behavior.  Some will do it, some won’t, and there’s no one to complain to!

You can, of course, do all kinds of work arounds like setup a filter to parse the request on your own, or even do it in the methods, but I hate work arounds, they have a tendency to fail at some point when things change.

In the end, my solution to cloudbees deployment was to get rid of PUT altogether, and use DELETE only in requests which do not need anything in the body.  While this will work, I am unhappy with this solution, to the point of thinking about going to EC2 and not working with Cloudbees!  My applications should work as required, and should not depend on the hosting platform’s limitations!!

UPDATE!

Following Michael’s comment (see below), I gave this item some more attention, and finally got it to work!

As it turns out, it IS possible to use PUT and DELETE in cloudbees, you just need to know how to make it work.  Since PUT requests are not parsed like POST, there is a trick to make the application side parse it correctly.

The way I did it, is by using @RequestBody to pass the data to my dto on the server side, setting my content type to application/json;charset=UTF-8 on the jQuery side, and using JSON.stringify to set the content correctly so it can be parsed by my JSON converter.

So, my method looked like so:

@RequestMapping(value=”{id}/status/read/}”, method=RequestMethod.PUT)
public ResponseEntity<String> markRead(
@CookieValue(value=Constants.AUTHENTICATED_USER_ID_COOKIE, required=false) UUIDValidatedString userIdCookie, 
@PathVariable UUIDValidatedString id,
@RequestBody TestDto test
) {

}
 
and my ajax side looked like so:
 
function internalMarkMessageReadStatus(messageId) {
var obj = new Object();
obj.test = “true”;
$.ajax({
url: ‘/api/messages/’ + messageId + ‘/status/read/’ ,
type: ‘PUT’,
dataType: ‘json’,
contentType: “application/json;charset=UTF-8”,
data: JSON.stringify(obj),
success: function(response) { 
}
});
}
 
The DTO is just a simple POJO:
 
package com.tipabout.dto;
import org.codehaus.jackson.annotate.JsonAutoDetect;
@Document
@JsonAutoDetect()
public class TestDto {
public String test;
 
public TestDto() {
}
 
public String getTest() {
return test;
}
 
public void setTest(String test) {
this.test = test;
}
}
 

Post 2 – The basics – Online hosting

August 19, 2012 Leave a comment

Well, I’m not a real expert at hosting solutions, but here’s what I’ve learned:

There are 4 possibilities when hosting online:

1. Buy cage space at any number of hosting spots.  They give you a cage, with electricity, network ,and air conditioning, and you have to do the rest (fill in hardware, cables, etc.).  You also get a service of a person who will go in and flip a power switch if you need it, so you don’t have to have someone on site).

I’ve been told that the costs start to balance out at around $1m, anything below that you should use the other options.  This is the cheapest solution if you have enough traffic and requirements to fill in the marginal cost, but is not for the average startup with $100k in the bank.

2. Use hosting solutions.  There are many, of all kinds, they vary in reliability and options, and are mid-range as far as costs are concerned.  A few months ago I would have said: use that if you want to get going as quick as possible, but now I know there are better options (see #4).  I guess the only reason to use that solution is if you are too big for option #3 (coming up), and too small for option #1.

3. Use PAAS such as EC2.  This is a very nice option, gives you endless scaling capabilities and easy deployment of (almost) any kind of environment, and best of all: you pay only for what you use, for as long as you use it.  You need 1000 Linux computers for the next 30 minutes?  setup one, duplicate 1000 times, and there you go.  Will cost you a couple of hundred bucks, but that’s it.  Amazing!  By the way, Amazon has free hosting for small environments so if you only want to play around, you don’t pay anything.  Haven’t played around with Azure, but I assume it works the same.

As far as I know this is the most expensive option, if you need an online solution which is constantly available and has high traffic and data requirements, I think you’ll balance out with option #2 in no time, and at some point will obviously get to #1 as well.

4. PAAS Level 2 (Cloudbees, Google apps engine, and the likes).  This is the best solution possible for startups at their early stages in my opinion.  I’ve been using Cloudbees for the past 2 months now, COMPLETELY FREE, and their solution rocks.  You get code hosting (I’m using GIT but you can also use SVN), as soon as you checkin your code and push it upstream to their repository (built into Eclipse if you use it), your code is copied over to Jenkins (I’ll get to CI in another post), and if all tests pass, your application is deployed in the cloud (on their app servers, I think it’s Tomcat).  You get access to a couple of DB’s (I’ve used MySQL and MongoDB) so you can save data, they have different monitoring solutions to complement, so in the end, you get a complete Continuous Deployment solution, with minimal effort and knowledge.

Just to be clear, I don’t work at Cloudbees, nor do I have any stocks, I just think what they did is absolutely phenomenal.  You can look at a youtube video which shows from zero to CD in 30 minutes using Cloudbees right here.  Check it out, it’s well worth your time.