Develop Reference

querying for records with date between two dates in mongo

I am having a problem where I am trying to find records with a particular username and where the start date is between a start and end date. I have searched around and below is the seemingly agreed upon answer for how to do this. Unfortunately when I use it I still get the same error:
Due to limitations of the com.mongodb.BasicDBObject, you can't add a second 'start' expression specified
The query I am attempting to use is:
Query query = new Query(
Criteria.where("username").is(username)
.andOperator(
Criteria.where("start").lt(DateUtils.ceiling(date)),
Criteria.where("start").gte(DateUtils.floor(date))
)
);
I am using spring-data-mongoldb 1.8.4.RELEASE

So this does in fact work properly. I just spent the last few hours ripping my code apart and rewriting it... Comparing with version in git shows no differences except now it works.
Just wanted to post this answer for anyone who comes looking. I really have no explanation as I was cleaning the project between runs, perhaps some copy of the old class was being held by the app server.

Writing one file per group in Pig Latin

The Problem:
I have numerous files that contain Apache web server log entries. Those entries are not in date time order and are scattered across the files. I am trying to use Pig to read a day's worth of files, group and order the log entries by date time, then write them to files named for the day and hour of the entries it contains.
Setup:
Once I have imported my files, I am using Regex to get the date field, then I am truncating it to hour. This produces a set that has the record in one field, and the date truncated to hour in another. From here I am grouping on the date-hour field.
First Attempt:
My first thought was to use the STORE command while iterating through my groups using a FOREACH and quickly found out that is not cool with Pig.
Second Attempt:
My second try was to use the MultiStorage() method in the piggybank which worked great until I looked at the file. The problem is that MulitStorage wants to write all fields to the file, including the field I used to group on. What I really want is just the original record written to the file.
The Question:
So...am I using Pig for something it is not intended for, or is there a better way for me to approach this problem using Pig? Now that I have this question out there, I will work on a simple code example to further explain my problem. Once I have it, I will post it here. Thanks in advance.

Out of the box, Pig doesn't have a lot of functionality. It does the basic stuff, but more times than not I find myself having to write custom UDFs or load/store funcs to get form 95% of the way there to 100% of the way there. I usually find it worth it since just writing a small store function is a lot less Java than a whole MapReduce program.
Your second attempt is really close to what I would do. You should either copy/paste the source code for MultiStorage or use inheritance as a starting point. Then, modify the putNext method to strip out the group value, but still write to that file. Unfortunately, Tuple doesn't have a remove or delete method, so you'll have to rewrite the entire tuple. Or, if all you have is the original string, just pull that out and output that wrapped in a Tuple.
Some general documentation on writing Load/Store functions in case you need a bit more help: http://pig.apache.org/docs/r0.10.0/udf.html#load-store-functions

Facebook Graph API does response any data when limit is set to 20

I'm querying data in the Facebook Graph API explorer:
access_token="SECRET"
GET https://graph.facebook.com/me/home?limit=20&until=1334555920&fields=id
result:
{
"data": [
]
}
I was shocked since there are many feeds on my "home".
Then I tried to set the limit to 100, then I got a feed list.
What's going on here? Does the "limit" parameter affect the graph api's result?
I tried to increase the limit to 25 and query again, there is one feed.
So what's the relationship between "limit" and "until"?

Facebook's API can be a little weird sometimes because of the data you're trying to access and there's a few parts to this question.
Limits
The limits are applied when data is returned, but before permissions and access controls are generated, which is explained with this blog post from last year: Limits in the Graph API.
Permissions
More importantly, even if you give yourself a token with every FB permission possible, you still won't be able to access everything that you created. Say you post something on a Friend's feed, but their feed is not set to Public privacy. Any queries against that friend's feed with your token will never return data (Or at least that was the case around a year ago).
API Itself
One of the most awesome bugs I found in the Graph API when I was working with it last year is the way it handles paging. The Graph API allows three filters: limit, offset, and since/until. Somewhere Facebook recommends (and rightly so) that you make use of the since/until dates exclusively whenever possible for paging. Ignoring debates as to why you would do that vs. offsets on a theoretical basis, on a practical one the following query used to degrade over time:
// This obviously isn't valid as written, but you the params change as described
limit=fixed-value&offset=programmatic-increase&since=some-fixed-date-here
The reason: Date ranges and offsets don't behave well with each other. As an example, say I made the following initial query:
// My example query
limit=20&since=1334555920
--> {#1,#2, ... #20}
Naturally you would want to page more data. The result would be something like this (I can't remember the exact pattern, but the top n would be repeats and the list of results would be truncated by n/2 or something similar):
// My example query
limit=20&since=1334555920&offset=20
---> {#10, #11 ... #25}
I never figured out why it happened, but eventually the query would taper off to return nothing and you would only get around 50-100 unique values. If you paged using dates exclusively however, you could go on for as long as the data would let you.
This is with the caveat that this was a bug and this was from a while ago. The main lesson here is I never would have found this bug without modifying my query to make things that should come out exactly the same (A particular date range based on posts #10-30 compared with a limit=20, offset=10) but the results were quite different.

Convert Large Switch to Hash (Or Other Recommendation)

I currently have an Android application that displays a schedule for a ferry boat. The application can display the full schedule (just a giant list), but the selling point in the application is it will display when the next two ferries are departing and how long from the current time that departure is.
I am relatively new to Java and currently use large Switch() statements in my code. Basically it gets the current phone time and compares it to all of the times in the schedule at which point it displays the next two departure times and then calculates the difference between current time and the departure times.
I am sure that a switch statement is not the best idea for speed purposes as well as code changing purposes. For example if one time changes its a bunch of lines of code to go in and fix for that one time change. Also if the entire schedule changes everyone has to update their app for the time change to take effect. My ideal situation would be to store a file somewhere on my webserver that could be downloaded and inserted into a hashmap (I think is the correct term) that would load the new schedule if there was a time change.
Not sure how confusing this is, but it would be greatly appreciated if someone could explain how I might use a hashmap or something else you might recommend to get this task accomplished. Currently the variables are the two ferry terminals as well as the day of the week since the schedule changes per day (monday, tues-friday, saturday, sunday).
Below is a screenshot of the application so you can understand it if my post wasn't clear. Thank you in advance.
Screenshot:

Store the schedule objects in a sorted array. You can then binary search the array for the first value greater than the current time. You'll probably use some parent array consisting of the location and applicable day of the week.
You can easily write that kind of data structure to a file that is read & parsed by the application for updates instead of being compiled into the code.
Details of this? First, understand resources in Android. If no updated schedule exists, fall back to the default resource.
Second, use an HTTP head request to check if a newer file exists. If it does, parse, download & save state. Saving Android Activity state using Save Instance State.
Finally, XML is handy for data distribution, even if it's not fast. Everybody understands it and it's easy to update or hand off.
<ferry location=0 time=2045>
<day>1</day>
<day>2</day>
<day>3</day>
<day>4</day>
<day>5</day>
</ferry>
<ferry location=0 time=0800>
<day>6</day>
</ferry>

You will need something like a database to hold the schedule data. That will help you to seperate code from data. I'm not familiar with Android but i think there is a interface to sqlite database on the device.
Further, as this is an application on a small device you may connect to the schedule database on a server thru the internet connection. That way you have to maintain schedule data only in one place (on the server) and clients will use always up to date data.

Where to start for my java program (Using the folder names to get info from IMDB)

I finished first year comp sci. And i want to spend some time working on the things they have taught us in the first year (lot of java and a bit of C)...
Anyways as a project, i wanted to do something i need, and what i need is a program to run through my movie folder and get the ratings and some basic info from IMDB...
I'm not sure where to start, i think i can handle the parts about reading the folder names, getting rid of the junk from the name to get the actual name and stuff.. Also i can handle the GUI but i don't know how i can talk to IMDB... what steps should i take to complete this project. I have about a month before school starts and i want to finish it before then...Thanks for all the input
EDIT:
Also can you guys tell me what i should start with and then move on to what? As in should i start with the GUI first or have the code that reads in the folder names and filters the names... I only wrote one program as an assignment in school and it was basically outlined step by step so i just wanna know what i should start with

You've made a very good start by decomposing the problem, identifying the kind of components you need and focusing on (an important) one that you don't know how to do.
The IMDB API is documented here and you can see that it amounts to sending simple HTTP requests with some paramters and getting back some formatted data, possibly as a JSON string.
You will find libraries to help with doing those two things. Even if there are public domain wrappers for accessing IMDB I'd recommend attempting to use general purpose HTTP and JSON libraries - that's probably a better educational exercise.

I'm the author of the IMDB API you are dicussing ;) I limit requests to 30 per hour to stop people hammering. I have yet to have a legitimate reason to perform more requests than that. My suggestion to anyone is to write a batch script to perform 1 request every 2minutes and then leave it going for a few hours overnight. Then you only have to perform a request on demand whenever you add a new movie.