I'm creating a UI system for an android game that will have a large (up to 4096x4096) background area in which menus can be placed anywhere within that screen and a camera will fly to that location when a different menu is needed. Instead of having a large static image, I'd like to be able to animate this slightly. What I'd like to know is how to do this efficiently without lagging up the device. These are the methods I've come up with so far, but maybe there is something better..
1) Have 3 separate 4096x4096 static layers for the background, 1 is the sky, one is the terrain, one is things like clouds and trees. Each layer is placed on top of each other with a slight difference in Z space to give a little parallax effect when the camera moves.
2) Have a large stationary background image, with a layer on top of that with individual specific sprites of clouds, trees and other things that should be animated. I think this might be the most efficient route, as I can choose not to animate parts that are not in view, but it will also limit re-usability as every different object will have to be placed manually in space. My goal is to be able to simply change the assets and be able to have a whole new game.
3) Have 1 large background layer with several frames that plays almost like a video. I feel like this will be the worst on performance(loading several 4096x4096 frames and drawing a different one 30 times a second), but would give me the scene exactly how I want it directly out of After Effects. I doubt this one is even feasible, not just because of the drawing but storage space on android devices just for the menu UI wouldn't allow for several 6MB frames.
Are any of these in the right direction? I have seen a few similar questions asked but none fit close enough to what I needed(A large, moving background that isn't made of tiles).
Any help is appreciated.
As far as your question is tagged for Android, I would recommend the 2nd solution.
The main reason is that solution #1 and #3 involve loading numerous 4096x4096 textures.
Quick calcultation: three 32bit textures with such resolution would use at least 200MB of Video RAM. It means that you can immediatly discard a lot of android devices.
On the other hand, the solution #2 would involve only two big textures: a large stationary background image, and a texture atlas containing specific sprites of clouds, trees...
This solution is really more memory friendly, and will lead to the same aestetic output.
TL;DR: the 3 solutions would work great but only the #2 would fit an embedded device
Related
I am using Java with OpenCV Library to detect Face,Eyes and Mouth using Laptop Camera.
What I have done so far:
Capture Video Frames using VideoCapture object.
Detect Face using Haar-Cascades.
Divide the Face region into Top Region and Bottom Region.
Search for Eyes inside Top region.
Search for Mouth inside Bottom region.
Problem I am facing:
At first Video is running normally and suddenly it becomes slower.
Main Questions:
Do Higher Cameras' Resolutions work better for Haar-Cascades?
Do I have to capture Video Frames in a certain scale? for example (100px X100px)?
Do Haar-Cascades work better in Gray-scale Images?
Does different lighting conditions make difference?
What does the method detectMultiScale(params) exactly do?
If I want to go for further analysis for Eye Blinking, Eye Closure Duration, Mouth Yawning, Head Nodding and Head Orientation to Detect Fatigue (Drowsiness) By Using Support Vector Machine, any advices?
Your help is appreciated!
The following article, would give you an overview of the things going under the hood, I would highly recommend to read the article.
Do Higher Cameras' Resolutions work better for Haar-Cascades?
Not necessarily, the cascade.detectMultiScale has params to adjust for various input width, height scenarios, like minSize and maxSize, These are optional params However, But you can tweak these to get robust predictions if you have control over the input image size. If you set the minSize to smaller value and ignore maxSize then it will work for smaller and high res images as well, but the performance would suffer. Also if you imagine now, How come there is no differnce between High-res and low-res images then you should consider that the cascade.detectMultiScale internally scales the images to lower resolutions for performance boost, that is why defining the maxSize and minSize is important to avoid any unnecessary iterations.
Do I have to capture Video Frames in a certain scale? for example
(100px X100px)
This mainly depends upon the params you pass to the cascade.detectMultiScale. Personally I guess that 100 x 100 would be too small for smaller face detection in the frame as some features would be completely lost while resizing the frame to smaller dimensions, and the cascade.detectMultiScale is highly dependent upon the gradients or features in the input image.
But if the input frame only has face as a major part, and there are no other smaller faces dangling behind then you may use 100 X 100. I have tested some sample faces of size 100 x 100 and it worked pretty well. And if this is not the case then 300 - 400 px width should work good. However you would need to tune the params in order to achieve accuracy.
Do Haar-Cascades work better in Gray-scale Images?
They work only in gray-scale images.
In the article, if you read the first part, you will come to know that it face detection is comprised of detecting many binary patterns in the image, This basically comes from the ViolaJones, paper which is the basic of this algorithm.
Does different lighting conditions make difference?
May be in some cases, largely Haar-features are lighting invariant.
If you are considering different lighting conditions as taking images under green or red light, then it may not affect the detection, The haar-features (since dependent on gray-scale) are independent of the RGB color of input image. The detection mainly depends upon the gradients/features in the input image. So as far as there are enough gradient differences in the input image such as eye-brow has lower intensity than fore-head, etc. it will work fine.
But consider a case when input image has back-light or very low ambient light, In that case it may be possible that some prominent features are not found, which may result in face not detected.
What does the method detectMultiScale(params) exactly do?
I guess, if you have read the article, by this time, then you must be knowing it well.
If I want to go for further analysis for Eye Blinking, Eye Closure
Duration, Mouth Yawning, Head Nodding and Head Orientation to Detect
Fatigue (Drowsiness) By Using Support Vector Machine, any advices?
No, I won't suggest you to perform these type of gesture detection with SVM, as it would be extremely slow to run 10 different cascades to conclude current facial state, However I would recommend you to use some Facial Landmark Detection Framework, such as Dlib, You may search for some other frameworks as well, because the model size of dlib is nearly 100MB and it may not suit your needs i f you want to port it to mobile device. So the key is ** Facial Landmark Detection **, once you get the full face labelled, you can draw conclusions like if the mouth if open or the eyes are blinking, and it works in Real-time, so your video processing won't suffer much.
I'm having quite a bit of difficulty wrapping my head around the actual display side of things with libgdx. That is, it just seems fairly jumbled in terms of what needs to be done in order to actually put something up onto the screen. I guess my confusion can sort of be separated into two parts:
What exactly needs to be done in terms of creating an image? There's
Texture, TextureRegion, TextureAtlas, Sprite, Batch, and probably a
few other art related assets that I'm missing. How do these all
relate and tie into each other? What's the "production chain" among
these I guess would be a way of putting it.
In terms of putting
whatever is created from the stuff above onto the monitor or
display, how do the different coordinate and sizing measures relate
and translate to and from each other? Say there's some image X that
I want to put on the screen. IT's got it's own set of dimensions and
coordinates, but then there's also a viewport size (is there a
viewport position?) and a camera position (is there a camera size?).
On top of all that, there's also the overall dispaly size that's
from Gdx.graphics. A few examples of things I might want to do could
be as follow:
X is my "global map" that is bigger than my screen
size. I want to be able to scroll/pan across it. What are the
coordinates/positions I should use when displaying it?
Y is bigger
than my screen size. I want to scale it down and have it always be
in the center of the screen/display. What scaling factor do I use
here, and which coordinates/positions?
Z is smaller than my screen
size. I want to stick it in the upper left corner of my screen and
have it "stick" to the global map I mentioned earlier. Which
positioning system do I use?
Sorry if that was a bunch of stuff... I guess the tl;dr of that second part is just which set of positions/coordinates, sizes, and scales am I supposed to do everything in terms of?
I know this might be a lot to ask at once, and I also know that most of this stuff can be found online, but after sifting through tutorial after tutorial, I can't seem to get a straight answer as to how these things all relate to each other. Any help would be appreciated.
Texture is essentially the raw image data.
TextureRegion allows you to grab smaller areas from a larger texture. For example, it is common practice to pack all of the images for your game/app into a single large texture (the LibGDX “TexturePacker” is a separate program that does this) and then use regions of the larger texture for your individual graphics. This is done because switching textures is a heavy and slow operation and you want to minimize this process.
When you pack your images into a single large image with the TexturePacker it creates a “.atlas” file which stores the names and locations of your individual images. TextureAtlas allows you to load the .atlas file and then extract your original images to use in your program.
Sprite adds position and color capabilities to the texture. Notice that the Texture API has no methods for setting/getting position or color. Sprites will be your characters and other objects that you can actually move around and position on the screen.
Batch/SpriteBatch is an efficient way of drawing multiple sprites to the screen. Instead of making drawing calls for each sprite one at a time the Batch does multiple drawing calls at once.
And hopefully I’m not adding to the confusion, but another I option I really like is using the “Actor” and “Stage” classes over the “Sprite” and “SpriteBatch” classes. Actor is similar to Sprite but adds additional functionality for moving/animating, via the act method. The Stage replaces the SpriteBatch as it uses its own internal SpriteBatch so you do not need to use the SpriteBatch explicitly.
There is also an entire set of UI components (table, button, textfield, slider, progress bar, etc) which are all based off of Actor and work with the Stage.
I can’t really help with question 2. I stick to UI-based apps, so I don’t know the best practices for working with large game worlds. But hopefully someone more knowledgeable in that area can help you with that.
This was to long to reply as a comment so I’m responding as another answer...
I think both Sprite/SpriteBatch and Actor/Stage are equally powerful as you can still animate and move with Sprite/SpriteBatch, but Actor/Stage is easier to work with. The stage has two methods called “act” and “draw” which allows the stage to update and draw every actor it contains very easily. You override the act method for each of your actors to specify what kind of action you want it to do. Look up a few different tutorials for Stage/Actor with sample code and it should become clear how to use it.
Also, I was slightly incorrect before that “Actor” is equivalent to Sprite, because Sprite includes a texture, but Actor by itself does not have any kind of graphical component. There is an extension of Actor called “Image” that includes a Drawable, so the Image class is actually the equivalent to Sprite. Actor is the base class that provides the methods for acting (or “updating”), but it doesn’t have to be graphical. I've used Actors for other purposes such as triggering audio sounds at specific times.
Atlas creates the large Texture containing all of your png files and then allows you to get regions from it for individual png's. So the pipeline for getting a specific png graphic would be Atlas > Region > Sprite/Image. Both Image and Sprite classes have constructors that take a region.
I am creating a Java game for Windows and I have come across a problem: there are lots of different screens and resolutions when it comes to Windows. What would be the best way to make it so that it looks just about the same on all screens?
You basically have three options:
Fix the size of the game window to something small that will fit n all screens (800*600 maybe)? This is easy to do, but could annoy users with big screens.....
Make the game resolution-independent, so that that it is rendered to a scale to fit the current window size. This is how most FPS games work for example. The main downside of this is that you need to do some extra scaling maths in your code and there may be some runtime overhead for rescaling images etc.
Make the game screen dynamically resizable, so that the components within it rearrange and resize themselves to fit the available space (like with a web page). This is the hardest to implement as you have to make use of appropriate layout managers and test lots of different combinations, but can give the nicest user "experience". I've successfully used MigLayout to do this in the past with a Swing game.
Any of these options could be best for you depending on the circumstances. It will probably depend mainly on the type/design of your game and your willingness to spend time on making the more complicated methods work well.
Here’s my task which I want to solve with as little effort as possible (preferrably with QT & C++ or Java): I want to use webcam video input to detect if there’s a (or more) crate(s) in front of the camera lens or not. The scene can change from "clear" to "there is a crate in front of the lens" and back while the cam feeds its video signal to my application. For prototype testing/ learning I have 2-3 images of the “empty” scene, and 2-3 images with one or more crates.
Do you know straightforward idea how to tackle this task? I found OpenCV, but isn't this framework too bulky for this simple task? I'm new to the field of computer vision. Is this generally a hard task or is it simple and robust to detect if there's an obstacle in front of the cam in live feeds? Your expert opinion is deeply appreciated!
Here's an approach I've heard of, which may yield some success:
Perform edge detection on your image to translate it into a black and white image, whereby edges are shown as black pixels.
Now create a histogram to record the frequency of black pixels in each vertical column of pixels in the image. The theory here is that a high frequency value in the histogram in or around one bucket is indicative of a vertical edge, which could be the edge of a crate.
You could also consider a second histogram to measure pixels on each row of the image.
Obviously this is a fairly simple approach and is highly dependent on "simple" input; i.e. plain boxes with "hard" edges against a blank background (preferable a background that contrasts heavily with the box).
You dont need a full-blown computer-vision library to detect if there is a crate or no crate in front of the camera. You can just take a snapshot and make a color-histogram (simple). To capture the snapshot take a look here:
http://msdn.microsoft.com/en-us/library/dd742882%28VS.85%29.aspx
Lots of variables here including any possible changes in ambient lighting and any other activity in the field of view. Look at implementing a Canny edge detector (which OpenCV has and also Intel Performance Primitives have as well) to look for the outline of the shape of interest. If you then kinda know where the box will be, you can perhaps sum pixels in the region of interest. If the box can appear anywhere in the field of view, this is more challenging.
This is not something you should start in Java. When I had this kind of problems I would start with Matlab (OpenCV library) or something similar, see if the solution would work there and then port it to Java.
To answer your question I did something similar by XOR-ing the 'reference' image (no crate in your case) with the current image then either work on the histogram (clustered pixels at right means large difference) or just sum the visible pixels and compare them with a threshold. XOR is not really precise but it is fast.
My point is, it took me 2hrs to install Scilab and the toolkits and write a proof of concept. It would have taken me two days in Java and if the first solution didn't work each additional algorithm (already done in Mat-/Scilab) another few hours. IMHO you are approaching the problem from the wrong angle.
If really Java/C++ are just some simple tools that don't matter then drop them and use Scilab or some other Matlab clone - prototyping and fine tuning would be much faster.
There are 2 parts involved in object detection. One is feature extraction, the other is similarity calculation. Some obvious features of the crate are geometry, edge, texture, etc...
So you can find some algorithms to extract these features from your crate image. Then comparing these features with your training sample images.
I am developing a 2D platformer game for the android platform, so I don't really care about the screen DPI, but much more about the actual resolution in pixels. From what I've gathered on the net, there are a couple of different resolutions (and aspect ratios) present. According to my search, the two resolutions that are currently widespread are 480x320 (1.5) and 800x480 (1.666), is that right? I'd like to target these two resolutions to reach most customers.
Now, I can deal with the different aspect ratios by showing a black border of 40 pixel for the bigger display, essentialy reducing it to 720x480 pixel and a ratio of 1.5.
The problem with my game is that it is essential for gameplay that the players see the same amount of the world on each screen. Otherwise, some players would get an unfair advantage. Furthermore, I trigger some events depending on the visibility. For example, an enemy is only allowed to start shooting when the player starts seeing it. Otherwise, the enemies' bullets would seem to come from nowhere.
So I figured I need to either create my graphics for one resolution and scale them for the other. Or I create separate graphics for each resolution. Is that right? Unfortunately, both ways are suboptimal for pixel graphics.
On another note: How can I restrict my game to these resolutions only (especially for the Android Market)? I know about the "supports-screens" tag in the manifest, but that works depending on the effective screen-size, not the size in pixel, or am I mistaken?
I am also interested in personal experiences of other android game developers when it comes to resolution independence.
Thanks!
My question would be: what do you think would do on a PC? For game development, Android should be looked at much more like a PC target than a console. You just intrinsically need to accept that there will be some diversity of screens that you can't totally predict up-front.
So I think there are two main approaches to take:
(1) Use a constant "display size" as if you were setting a fixed video resolution on the PC and letting the user's monitor deal with it. On these devices of course there is no monitor, just one fixed display, so it doesn't make sense to modify the core resolution. Instead, you can set up the SurfaceView showing your game to have a fixed resolution, and let the platform's compositor take care of scaling it (in hardware) as it composites to the screen.
(2) More intelligently adjust to the actual resolution of the screen you find yourself running in. Scale up or down graphics yourself to create the playing area you want. Maybe have some different sizes of textures and select the appropriate ones for the screen resolution.
You could probably also do a combination of these, where you have a couple fixed sizes you pick for the surface view depending on the total resolution available which the game can run well with. In either case, you can do letter-boxing as appropriate to keep your aspect ratio constant on different screens, if that is what you want.
There are three approaches to differences in aspect ratio:
Show opaque borders on some ratios ("letterboxing").
Show more of the game world on some ratios.
Don't work at all on some ratios.
With approach (1) you waste screen space on some devices. Not such a big deal for televisions, but miserable on handheld devices where screen space is limited. With approach (2) players on some devices get advantages (they can see more of the world) and disadvantages (sprites are smaller, so touch precision is harder). Approach (3) just sucks.
Obviously it depends on the details of your game which is better, but as a player I much prefer approach (2). The constituency who care if players on other devices get a bit of a hypothetical advantage is pretty small compared to the constituency who care if their screen is partly obscured by unnecessary black bars.
(Similar approaches and remarks apply to differences in resolution.)