Development and deployment of Spark applications with Scala, Eclipse, and sbt – Part 1: Installation & configuration

Constantinos Voglis Big Data, Spark 22 Comments

The purpose of this tutorial is to setup the necessary environment for development and deployment of Spark applications with Scala. Specifically, we are going to use the Eclipse IDE for development of applications and deploy them with spark-submit. The glue that ties everything together is the sbt interactive build tool. The sbt tool provides plugins used to: Create an Eclipse …

Maven add dependencies from central repository to ADF project in JDeveloper

Chris Vezalis DEVOPS, Maven, Oracle ADF 3 Comments

ADF Projects often need jars from popular open source libraries and frameworks. JDeveloper supports the addition of dependencies from public (and private) repositories. To add a new dependency to project we need to: Go to Project folder in Application browser and expand the Resources folder under Project. Then open pom.xml file and go to dependencies section. The click the Green …

Deck Transition changes Checkbox’s value (Oracle Jdeveloper 12.1.3.0)

Rigas Papazisis Fusion Middleware, Oracle ADF Leave a Comment

A really nice new feature introduced in ADF 12c is the deck ui component. Two helpful examples on how to use this component are the following: How to use the Deck component (ADF 12.1.3 New Feature) from Andreas Koop Using af:deck component to animate content in ADF 12c(12.1.3) from Ashish Awasthi You can find some more examples over the internet but …

Unexpected behavior of Spark dataframe filter method

Christos - Iraklis Tsatsoulis Big Data, Spark 4 Comments

[EDIT: Thanks to this post, the issue reported here has been resolved since Spark 1.4.1 – see the comments below] While writing the previous post on Spark dataframes, I encountered an unexpected behavior of the respective .filter method; but, on the one hand, I needed some more time to experiment and confirm it and, on the other hand, I knew …

Installing rJava R package in Oracle Linux

Christos - Iraklis Tsatsoulis R 1 Comment

The Oracle Big Data Lite (BDLite) VM is a handy and convenient platform for testing, development, and training on the related tools and technologies, such as Cloudera Hadoop, Oracle NoSQL database, Oracle SQL Developer & Data Modeler etc. Among other things, it includes a full distribution of the Oracle R Enterprise (ORE) and the Oracle R Connectors for Hadoop (ORCH). …

ADF Calendar Implementation

Gabriel Sideras Oracle ADF 2 Comments

One of the new projects that we are currently working on is the implementation of a complete school management solution using Oracle ADF 12c. As one might imagine, one of the basic requirements of this project was to implement a Scheduling functionality. The schedule implementation has many different aspects, but in this post we will focus on the ADF Calendar implementation that will …

Spark data frames from CSV files: handling headers & column types

Christos - Iraklis Tsatsoulis Big Data, Spark 14 Comments

If you come from the R (or Python/pandas) universe, like me, you must implicitly think that working with CSV files must be one of the most natural and straightforward things to happen in a data analysis context. Indeed, if you have your data in a CSV file, practically the only thing you have to do from R is to fire …

ViewCriteria issue when using same attribute twice (Oracle JDeveloper 12.1.3.0)

Rigas Papazisis Fusion Middleware, Oracle ADF 1 Comment

Some days ago, in an Oracle MAF technical workshop, I witnessed one of those awkward moments when you are certain that something should work correctly in a presentation but it didn’t. While showing us some basic functionality in Oracle ADF 12c, the instructor implemented a simple query using ViewCriteria. He used an OR conjunction but the query seemed to react …

Undocumented behavior of ore.make.names() function in Oracle R Enterprise

Christos - Iraklis Tsatsoulis Oracle R 2 Comments

While working with some data in Hive recently using the Oracle R Connectors for Hadoop (ORCH), I tried to use the ore.make.names function (of package OREbase ). The function creates valid column names for ore.frame objects. Here is a reproducible example, copied straight from the function documentation: Experimenting a little, I discovered that ore.make.names becomes functional after executing ore.connect. Indeed, …