Android Parsing YQL Using JSON Tutorial
Last Updated: May 14, 2013
In this tutorial, you will learn how to parse YQL (Yahoo Query Language) using JSON in your Android application. YQL Web Service accesses a datasource on the Internet, transforms the data, and returns the results in either XML or JSON format. YQL can access several types of web content formats such as HTML, XML, RSS, and Atom. We will parse YQL using JSON into a ListView, and on ListView item click will show results on a new activity. So lets begin…
Create a new project in Eclipse File > New > Android Application Project. Fill in the details and name your project YQLTutorial.
Application Name : YQLTutorial
Project Name : YQLTutorial
Package Name : com.androidbegin.yqltutorial
Open your MainActivity.java and paste the following code.
MainActivity.java
package com.androidbegin.yqltutorial; import java.util.ArrayList; import java.util.HashMap; import org.json.JSONArray; import org.json.JSONException; import org.json.JSONObject; import android.app.Activity; import android.app.ProgressDialog; import android.os.AsyncTask; import android.os.Bundle; import android.util.Log; import android.widget.ListView; public class MainActivity extends Activity { // Declare Variables ListView listview; ListViewAdapter adapter; ProgressDialog mProgressDialog; ArrayList<HashMap<String, String>> arraylist; static String TITLE = "title"; static String DESC = "description"; static String THUMB = "thumbnail"; @Override public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); // Get the view from listview_main.xml setContentView(R.layout.listview_main); // Execute DownloadJSON AsyncTask new DownloadJSON().execute(); } // DownloadJSON AsyncTask private class DownloadJSON extends AsyncTask<Void, Void, Void> { @Override protected void onPreExecute() { super.onPreExecute(); // Create a progressdialog mProgressDialog = new ProgressDialog(MainActivity.this); // Set progressdialog title mProgressDialog.setTitle("Android Parsing YQL in JSON Tutorial"); // Set progressdialog message mProgressDialog.setMessage("Loading..."); mProgressDialog.setIndeterminate(false); // Show progressdialog mProgressDialog.show(); } @Override protected Void doInBackground(Void... params) { // Create the array arraylist = new ArrayList<HashMap<String, String>>(); // YQL JSON URL String url = "http://query.yahooapis.com/v1/public/yql?q=SELECT%20*%20FROM%20google.books%20WHERE%20q%3D%22android%22%20AND%20maxResults%3D5%20AND%20startIndex%3D1&format=json&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys&callback="; try { // Retrive JSON Objects from the given URL in JSONfunctions.class JSONObject json_data = JSONfunctions.getJSONfromURL(url); JSONObject json_query = json_data.getJSONObject("query"); JSONObject json_results = json_query.getJSONObject("results"); JSONObject json_json_result = json_results .getJSONObject("json"); JSONArray json_result = json_json_result.getJSONArray("items"); for (int i = 0; i < json_result.length(); i++) { HashMap<String, String> map = new HashMap<String, String>(); JSONObject c = json_result.getJSONObject(i); JSONObject vo = c.getJSONObject("volumeInfo"); map.put("title", vo.optString("title")); map.put("description", vo.optString("description")); JSONObject il = vo.getJSONObject("imageLinks"); map.put("thumbnail", il.optString("thumbnail")); arraylist.add(map); } } catch (JSONException e) { Log.e("Error", e.getMessage()); e.printStackTrace(); } return null; } @Override protected void onPostExecute(Void args) { // Locate the listview in listview_main.xml listview = (ListView) findViewById(R.id.listview); // Pass the results into ListViewAdapter.java adapter = new ListViewAdapter(MainActivity.this, arraylist); // Binds the Adapter to the ListView listview.setAdapter(adapter); // Close the progressdialog mProgressDialog.dismiss(); } } }
We have created a ListView and implemented an AsyncTask class for retrieval of JSON Objects and set it into string arrays. Then send the string arrays to a ListViewAdapter. Below is our YQL QUERY URL from the YQL Console.
YQL QUERY URL LINK
http://query.yahooapis.com/v1/public/yql?q=SELECT%20*%20FROM%20google.books%20WHERE%20q%3D%22android%22%20AND%20maxResults%3D5%20AND%20startIndex%3D1&format=json&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys&callback=
You can check out link above to see the results. The data is from google books search and is converted into a JSON data-interchange format. Follow the steps below to learn how to get the YQL QUERY URL.
YQL CONSOLE
Go to YQL Console
INSERT YQL STATEMENT
Type into the YQL Statement textbox : SELECT * FROM google.books WHERE q=”android” AND maxResults=5 AND startIndex=1
Tick the JSON radiobutton, untick Diagnostics CheckBox and remove “cbfunc” in the textbox.
Click on TEST and you should get the JSON results shown on the screenshot below.
The YQL QUERY URL is in the REST QUERY TextBox.
Next, create an XML graphical layout for your MainActivity. Go to res > layout > Right Click on layout > New > Android XML File|
Name your new XML file listview_main.xml and paste the following code.
listview_main.xml
<?xml version="1.0" encoding="utf-8"?> <RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android" android:layout_width="fill_parent" android:layout_height="fill_parent" > <ListView android:id="@+id/listview" android:layout_width="fill_parent" android:layout_height="fill_parent" /> </RelativeLayout>
Output:
Next, create a JSON function class. Go to File > New > Class and name it JSONfuntions.java. Select your package named com.androidbegin.yqltutorial and click Finish.
Open your JSONfuntions.java and paste the following code.
JSONfuntions.java
package com.example.yqltutorial; import java.io.BufferedReader; import java.io.InputStream; import java.io.InputStreamReader; import org.apache.http.HttpEntity; import org.apache.http.HttpResponse; import org.apache.http.client.HttpClient; import org.apache.http.client.methods.HttpPost; import org.apache.http.impl.client.DefaultHttpClient; import org.json.JSONException; import org.json.JSONObject; import android.util.Log; public class JSONfunctions { public static JSONObject getJSONfromURL(String url){ InputStream is = null; String result = ""; JSONObject jArray = null; // Download JSON data from URL try{ HttpClient httpclient = new DefaultHttpClient(); HttpPost httppost = new HttpPost(url); HttpResponse response = httpclient.execute(httppost); HttpEntity entity = response.getEntity(); is = entity.getContent(); }catch(Exception e){ Log.e("log_tag", "Error in http connection "+e.toString()); } // Convert response to string try{ BufferedReader reader = new BufferedReader(new InputStreamReader(is,"iso-8859-1"),8); StringBuilder sb = new StringBuilder(); String line = null; while ((line = reader.readLine()) != null) { sb.append(line + "\n"); } is.close(); result=sb.toString(); }catch(Exception e){ Log.e("log_tag", "Error converting result "+e.toString()); } try{ jArray = new JSONObject(result); }catch(JSONException e){ Log.e("log_tag", "Error parsing data "+e.toString()); } return jArray; } }
This class downloads the JSON file and checks whether it is built accordingly.
Next, create a ListViewAdapter class. Go to File > New > Class and name it ListViewAdapter.java. Select your package named com.androidbegin.yqltutorial and click Finish.
Open your ListViewAdapter.java and paste the following code.
ListViewAdapter.java
package com.androidbegin.yqltutorial; import java.util.ArrayList; import java.util.HashMap; import android.content.Context; import android.content.Intent; import android.view.LayoutInflater; import android.view.View; import android.view.View.OnClickListener; import android.view.ViewGroup; import android.widget.BaseAdapter; import android.widget.ImageView; import android.widget.TextView; public class ListViewAdapter extends BaseAdapter { // Declare Variables Context context; LayoutInflater inflater; ArrayList<HashMap<String, String>> data; ImageLoader imageLoader; public ListViewAdapter(Context context, ArrayList<HashMap<String, String>> arraylist) { this.context = context; data = arraylist; imageLoader = new ImageLoader(context); } @Override public int getCount() { return data.size(); } @Override public Object getItem(int position) { return null; } @Override public long getItemId(int position) { return 0; } public View getView(final int position, View convertView, ViewGroup parent) { // Declare Variables TextView txttitle; TextView txtdesc; ImageView thumb; inflater = (LayoutInflater) context .getSystemService(Context.LAYOUT_INFLATER_SERVICE); View itemView = inflater.inflate(R.layout.listview_item, parent, false); // Get the position from the results HashMap<String, String> resultp = new HashMap<String, String>(); resultp = data.get(position); // Locate the TextView in listview_item.xml txttitle = (TextView) itemView.findViewById(R.id.title); txtdesc = (TextView) itemView.findViewById(R.id.desc); // Locate the ImageView in listview_item.xml thumb = (ImageView) itemView.findViewById(R.id.thumb); // Capture position and set results to the TextViews txttitle.setText(resultp.get(MainActivity.TITLE)); txtdesc.setText(resultp.get(MainActivity.DESC)); // Capture position and set results to the ImageView // Passes thumbnail images URL into ImageLoader.class to download and // cache images imageLoader.DisplayImage(resultp.get(MainActivity.THUMB), thumb); // Capture button clicks on ListView items itemView.setOnClickListener(new OnClickListener() { @Override public void onClick(View arg0) { // Get the position from the results HashMap<String, String> resultp = new HashMap<String, String>(); resultp = data.get(position); // Send single item click data to SingleItemView Class Intent intent = new Intent(context, SingleItemView.class); // Pass all data title intent.putExtra("title", resultp.get(MainActivity.TITLE)); // Pass all data description intent.putExtra("description", resultp.get(MainActivity.DESC)); // Pass all data thumbnail intent.putExtra("thumbnail", resultp.get(MainActivity.THUMB)); // Start SingleItemView Class context.startActivity(intent); } }); return itemView; } }
In this custom listview adapter class, string arrays are passed into the ListViewAdapter and set into the TextViews and ImageViews followed by the positions. On listview item click will pass the string arrays and position to a new activity.
Next, create an XML graphical layout for your listview item. Go to res > layout > Right Click on layout > New > Android XML File
Name your new XML file listview_item.xml and paste the following code.
listview_item.xml
<?xml version="1.0" encoding="utf-8"?> <RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android" android:layout_width="fill_parent" android:layout_height="fill_parent" > <ImageView android:id="@+id/thumb" android:layout_width="wrap_content" android:layout_height="wrap_content" android:layout_alignParentLeft="true" android:padding="10dp" /> <TextView android:id="@+id/title" android:layout_width="wrap_content" android:layout_height="wrap_content" android:layout_toRightOf="@+id/thumb" /> <TextView android:id="@+id/desc" android:layout_width="wrap_content" android:layout_height="wrap_content" android:layout_below="@+id/title" android:layout_toRightOf="@+id/thumb" /> </RelativeLayout>
Next, create an imageloader class. Go to File > New > Class and name it ImageLoader.java. Select your package named com.androidbegin.yqltutorial and click Finish.
Open your ImageLoader.java and paste the following code.
ImageLoader.java
package com.example.yqltutorial; import java.io.File; import java.io.FileInputStream; import java.io.FileNotFoundException; import java.io.FileOutputStream; import java.io.IOException; import java.io.InputStream; import java.io.OutputStream; import java.net.HttpURLConnection; import java.net.URL; import java.util.Collections; import java.util.Map; import java.util.WeakHashMap; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; import android.os.Handler; import android.content.Context; import android.graphics.Bitmap; import android.graphics.BitmapFactory; import android.widget.ImageView; public class ImageLoader { MemoryCache memoryCache = new MemoryCache(); FileCache fileCache; private Map<ImageView, String> imageViews = Collections .synchronizedMap(new WeakHashMap<ImageView, String>()); ExecutorService executorService; // Handler to display images in UI thread Handler handler = new Handler(); public ImageLoader(Context context) { fileCache = new FileCache(context); executorService = Executors.newFixedThreadPool(5); } final int stub_id = R.drawable.temp_image; public void DisplayImage(String url, ImageView imageView) { imageViews.put(imageView, url); Bitmap bitmap = memoryCache.get(url); if (bitmap != null) imageView.setImageBitmap(bitmap); else { queuePhoto(url, imageView); imageView.setImageResource(stub_id); } } private void queuePhoto(String url, ImageView imageView) { PhotoToLoad p = new PhotoToLoad(url, imageView); executorService.submit(new PhotosLoader(p)); } private Bitmap getBitmap(String url) { File f = fileCache.getFile(url); Bitmap b = decodeFile(f); if (b != null) return b; // Download Images from the Internet try { Bitmap bitmap = null; URL imageUrl = new URL(url); HttpURLConnection conn = (HttpURLConnection) imageUrl .openConnection(); conn.setConnectTimeout(30000); conn.setReadTimeout(30000); conn.setInstanceFollowRedirects(true); InputStream is = conn.getInputStream(); OutputStream os = new FileOutputStream(f); Utils.CopyStream(is, os); os.close(); conn.disconnect(); bitmap = decodeFile(f); return bitmap; } catch (Throwable ex) { ex.printStackTrace(); if (ex instanceof OutOfMemoryError) memoryCache.clear(); return null; } } // Decodes image and scales it to reduce memory consumption private Bitmap decodeFile(File f) { try { // Decode image size BitmapFactory.Options o = new BitmapFactory.Options(); o.inJustDecodeBounds = true; FileInputStream stream1 = new FileInputStream(f); BitmapFactory.decodeStream(stream1, null, o); stream1.close(); // Find the correct scale value. It should be the power of 2. // Recommended Size 512 final int REQUIRED_SIZE = 512; int width_tmp = o.outWidth, height_tmp = o.outHeight; int scale = 1; while (true) { if (width_tmp / 2 < REQUIRED_SIZE || height_tmp / 2 < REQUIRED_SIZE) break; width_tmp /= 2; height_tmp /= 2; scale *= 2; } // Decode with inSampleSize BitmapFactory.Options o2 = new BitmapFactory.Options(); o2.inSampleSize = scale; FileInputStream stream2 = new FileInputStream(f); Bitmap bitmap = BitmapFactory.decodeStream(stream2, null, o2); stream2.close(); return bitmap; } catch (FileNotFoundException e) { } catch (IOException e) { e.printStackTrace(); } return null; } // Task for the queue private class PhotoToLoad { public String url; public ImageView imageView; public PhotoToLoad(String u, ImageView i) { url = u; imageView = i; } } class PhotosLoader implements Runnable { PhotoToLoad photoToLoad; PhotosLoader(PhotoToLoad photoToLoad) { this.photoToLoad = photoToLoad; } @Override public void run() { try { if (imageViewReused(photoToLoad)) return; Bitmap bmp = getBitmap(photoToLoad.url); memoryCache.put(photoToLoad.url, bmp); if (imageViewReused(photoToLoad)) return; BitmapDisplayer bd = new BitmapDisplayer(bmp, photoToLoad); handler.post(bd); } catch (Throwable th) { th.printStackTrace(); } } } boolean imageViewReused(PhotoToLoad photoToLoad) { String tag = imageViews.get(photoToLoad.imageView); if (tag == null || !tag.equals(photoToLoad.url)) return true; return false; } // Used to display bitmap in the UI thread class BitmapDisplayer implements Runnable { Bitmap bitmap; PhotoToLoad photoToLoad; public BitmapDisplayer(Bitmap b, PhotoToLoad p) { bitmap = b; photoToLoad = p; } public void run() { if (imageViewReused(photoToLoad)) return; if (bitmap != null) photoToLoad.imageView.setImageBitmap(bitmap); else photoToLoad.imageView.setImageResource(stub_id); } } public void clearCache() { memoryCache.clear(); fileCache.clear(); } }
An imageloader is class that helps you download, display and cache images. By using an imageloader, images will be unloaded automatically if the device memory is low and it makes sure that the images are sized appropriately, and cached in the memory. Insert a temporary image for the imageloader to display when an image is unavailable or its still loading. For this tutorial, we have prepared a sample temporary image. Insert your downloaded sample image into your res > drawable-hdpi.
Temporary Image
[wpfilebase tag=file id=32 tpl=download-button /]
Next, create a memory cache class. Go to File > New > Class and name it MemoryCache.java. Select your package named com.androidbegin.yqltutorial and click Finish.
Open your MemoryCache.java and paste the following code.
MemoryCache.java
package com.example.yqltutorial; import java.util.Collections; import java.util.Iterator; import java.util.LinkedHashMap; import java.util.Map; import java.util.Map.Entry; import android.graphics.Bitmap; import android.util.Log; public class MemoryCache { private static final String TAG = "MemoryCache"; // Last argument true for LRU ordering private Map<String, Bitmap> cache = Collections .synchronizedMap(new LinkedHashMap<String, Bitmap>(10, 1.5f, true)); // Current allocated size private long size = 0; // Max memory in bytes private long limit = 1000000; public MemoryCache() { // Use 25% of available heap size setLimit(Runtime.getRuntime().maxMemory() / 4); } public void setLimit(long new_limit) { limit = new_limit; Log.i(TAG, "MemoryCache will use up to " + limit / 1024. / 1024. + "MB"); } public Bitmap get(String id) { try { if (!cache.containsKey(id)) return null; return cache.get(id); } catch (NullPointerException ex) { ex.printStackTrace(); return null; } } public void put(String id, Bitmap bitmap) { try { if (cache.containsKey(id)) size -= getSizeInBytes(cache.get(id)); cache.put(id, bitmap); size += getSizeInBytes(bitmap); checkSize(); } catch (Throwable th) { th.printStackTrace(); } } private void checkSize() { Log.i(TAG, "cache size=" + size + " length=" + cache.size()); if (size > limit) { // Least recently accessed item will be the first one iterated Iterator<Entry<String, Bitmap>> iter = cache.entrySet().iterator(); while (iter.hasNext()) { Entry<String, Bitmap> entry = iter.next(); size -= getSizeInBytes(entry.getValue()); iter.remove(); if (size <= limit) break; } Log.i(TAG, "Clean cache. New size " + cache.size()); } } public void clear() { try { cache.clear(); size = 0; } catch (NullPointerException ex) { ex.printStackTrace(); } } long getSizeInBytes(Bitmap bitmap) { if (bitmap == null) return 0; return bitmap.getRowBytes() * bitmap.getHeight(); } }
This memory cache class will limit the memory usage when loading images. Which means, the images will be removed if not shown within the content view.
Next, create a file cache class. Go to File > New > Class and name it FileCache.java. Select your package named com.androidbegin.yqltutorial and click Finish.
Open your FileCache.java and paste the following code.
FileCache.java
package com.example.yqltutorial; import java.io.File; import android.content.Context; public class FileCache { private File cacheDir; public FileCache(Context context) { // Find the dir to save cached images if (android.os.Environment.getExternalStorageState().equals( android.os.Environment.MEDIA_MOUNTED)) cacheDir = new File( android.os.Environment.getExternalStorageDirectory(), "YQLTutorialCache"); else cacheDir = context.getCacheDir(); if (!cacheDir.exists()) cacheDir.mkdirs(); } public File getFile(String url) { String filename = String.valueOf(url.hashCode()); // String filename = URLEncoder.encode(url); File f = new File(cacheDir, filename); return f; } public void clear() { File[] files = cacheDir.listFiles(); if (files == null) return; for (File f : files) f.delete(); } }
This file cache class saves temporary images into the device internal storage to prevent the images to be downloaded repeatedly.
Next, create an utility class. Go to File > New > Class and name it Utils.java. Select your package named com.androidbegin.yqltutorial and click Finish.
Open your Utils.java and paste the following code.
Utils.java
package com.example.yqltutorial; import java.io.InputStream; import java.io.OutputStream; public class Utils { public static void CopyStream(InputStream is, OutputStream os) { final int buffer_size = 1024; try { byte[] bytes = new byte[buffer_size]; for (;;) { int count = is.read(bytes, 0, buffer_size); if (count == -1) break; os.write(bytes, 0, count); } } catch (Exception ex) { } } }
Next, create an activity to display results. Go to File > New > Class and name it SingleItemView.java. Select your package named com.androidbegin.yqltutorial and click Finish.
Open your SingleItemView.java and paste the following code.
SingleItemView.java
package com.androidbegin.yqltutorial; import android.app.Activity; import android.content.Intent; import android.os.Bundle; import android.widget.ImageView; import android.widget.TextView; public class SingleItemView extends Activity { // Declare Variables String title; String description; String thumbnail; ImageLoader imageLoader = new ImageLoader(this); @Override public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); // Get the view from singleitemview.xml setContentView(R.layout.singleitemview); Intent i = getIntent(); // Get the result of title title = i.getStringExtra("title"); // Get the result of description description = i.getStringExtra("description"); // Get the result of thumbnail thumbnail = i.getStringExtra("thumbnail"); // Locate the TextView in singleitemview.xml TextView txttitle = (TextView) findViewById(R.id.title); TextView txtdescription = (TextView) findViewById(R.id.description); // Locate the ImageView in singleitemview.xml ImageView img = (ImageView) findViewById(R.id.thumbnail); // Set results to the TextView txttitle.setText(title); txtdescription.setText(description); imageLoader.DisplayImage(thumbnail, img); } }
In this activity, strings are retrieved from the ListViewAdapter by using Intent and sets into the TextViews and an image URL into ImageLoader class to load images into the ImageView.
Next, create an XML graphical layout for your SingleItemView. Go to res > layout > Right Click on layout > New > Android XML File
Name your new XML file singleitemview.xml and paste the following code.
singleitemview.xml
<?xml version="1.0" encoding="utf-8"?> <RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android" android:layout_width="fill_parent" android:layout_height="fill_parent" > <ImageView android:id="@+id/thumbnail" android:layout_width="wrap_content" android:layout_height="wrap_content" android:layout_alignParentLeft="true" android:padding="10dp" /> <TextView android:id="@+id/title" android:layout_width="wrap_content" android:layout_height="wrap_content" android:layout_toRightOf="@+id/thumbnail" /> <TextView android:id="@+id/description" android:layout_width="wrap_content" android:layout_height="wrap_content" android:layout_toRightOf="@+id/thumbnail" android:layout_below="@+id/title" /> </RelativeLayout>
Next, change the application name and texts. Open your strings.xml in your res > values folder and paste the following code.
strings.xml
<resources> <string name="app_name">YQL Tutorial</string> <string name="hello_world">Hello world!</string> <string name="menu_settings">Settings</string> <string name="title_activity_main">YQL Tutorial</string> </resources>
In your AndroidManifest.xml, we need to declare permissions to allow the application to write an external storage and connect to the Internet. Open your AndroidManifest.xml and paste the following code.
AndroidManifest.xml
<manifest xmlns:android="http://schemas.android.com/apk/res/android" package="com.example.yqltutorial" android:versionCode="1" android:versionName="1.0" > <uses-sdk android:minSdkVersion="8" android:targetSdkVersion="15" /> <uses-permission android:name="android.permission.INTERNET" /> <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" /> <application android:icon="@drawable/ic_launcher" android:label="@string/app_name" android:theme="@style/AppTheme" > <activity android:name=".MainActivity" android:label="@string/title_activity_main" > <intent-filter> <action android:name="android.intent.action.MAIN" /> <category android:name="android.intent.category.LAUNCHER" /> </intent-filter> </activity> <activity android:name=".SingleItemView" > </activity> </application> </manifest>
Output:
Source Code :
[purchase_link id=”7930″ text=”Purchase to Download Source Code” style=”button” color=”green”]/]
Hi guys, I just found out that the "The Twitter REST API v1 is no longer active". It seems Twitter do not allow public query anymore. I will have to rewrite this post. For more info on Twitter REST API visit this link https://dev.twitter.com/docs/api/1.1/overview
AndroidBegin
Android Parsing YQL Using JSON Tutorial