Friday, October 25, 2024

Installing OAS on mobile laptop

Once in a while I install OAS (Oracle Analytics Server) on my laptop. I do it for testing purposes mostly... I don't want anyone but myself to connect to this OAS. Sort of rich man's OA Desktop.

 First of all, don't do it. Even if you want OAS on a laptop, you would better use a VM or Docker. This type of installation is not supported and usually is problematic.

This post is for myself, so I will stop reinventing the wheel each time I do it...

Problem: 

  • OAS is a server that requires a static IP. 
  • My laptop changes the IPs each time I move from one network to another. 
  • During the installation the IP is hard coded in some of the config files.

Work around: 

  • Create a MS loopback adapter with a static IP. Example from Oracle DB 21c documentation is here.
  • Set that IP as Ethernet network on the laptop.
  • Before OAS installation turn of Wi-Fi or any other network, so the installation is forced to use the loopback IP.

Once done remember to stop and start OAS on the same network.

Thursday, October 24, 2024

Oracle OAS 2024 (7.6) October 2024 patch and the missing OPATCH

 Oracle has released the Oracle Analytics Server - OAS 2024 (7.6) October 2024 bundle patch, specifically 09-Oct-2024 patch. You can find it at the regular place: Doc ID 2832967.2 in the OAS tab.



The patch requires upgrade of OPatch,  at least to version 13.9.4.2.13. For some reason I couldn't find the relevant OPatch version at the regular place (https://updates.oracle.com/download/6880880.html) where the FMW patch is described as Obsolete, (correct for 24 Oct. 2024).

You can visit this support note (Patch 28186730: OPATCH 13.9.4.2.17 FOR EM 13.5 AND FMW/WLS 12.2.1.4.0 AND 14.1.1.0.0) to download OPatch 13.9.4.2.17 that should do the trick.



Tuesday, June 4, 2024

Oracle Spatial - Millions of Points in Thousands of Polygons

I wrote about Oracle Spatial in the past. (Intro, data loading Points, Lines, Polygons). Now I had a new challenge.

We had a POC. Customer wanted to test performance of 7 million data points in 3200 polygons. In the POC I learned there is difference in performance, based on the way we create the query. I was lucky to gain some guidance from Daniel Geringer (who wrote this guide) of Oracle Spatial. Bellow is the summary of the process. It was done on Autonomous Database (ADW), but should be similar on any Oracle DB.

First part will cover the performance benefits from using the proper function. Second, some advisable data setting and cleaning steps we did. 


Part 1 

Customer loaded ~7 million data points (lng/lat) of USA accidents from a similar data source and loaded to ADW. Loaded ~3200 US counties from this or similar source

We could update the longitude and latitude (lng/lat) data and place them in a new spatial format column. For many reasons we preferred not to.  We worked directly with the lng/lat points and the get_geometry function (see page 25 here or slightly longer version here). Next created index (see pages 26, 27 here)... and fixed some inconsistencies (see second part, bellow).

My default intention was to use the SDO_CONTAINS function. Instead of SDO_CONTAINS, Daniel suggested top use  SDO_ANYINTERACT or SDO_INSIDE instead and to switch the order of the geometry parameter to the spatial operator. 

SDO_ANYINTERACT can be used if it's ok to return points on the polygon boundary.  If the requirement is only to return points inside the polygon, SDO_INSIDE should be used. In switching the geometry order, there is a huge difference between running 3200 times comparisons for 7 million points and running 7 million times comparisons for 3200 polygons. In our tests it was ~390 seconds vs. ~5 seconds.

In all cases the following format was used:

select count(*) from accident a, counties c where SDO_... ='TRUE';

Let's compare the following:

  • SDO_CONTAINS(C.GEOMETRY, GET_GEOMETRY(LNG,LAT)) = 'TRUE'
  • SDO_ANYINTERACT(GET_GEOMETRY(LNG,LAT),c.geometry)='TRUE'
  • SDO_ANYINTERACT(c.geometry,GET_GEOMETRY(LNG,LAT))='TRUE'
  • SDO_INSIDE(GET_GEOMETRY(LNG,LAT),c.geometry)='TRUE'

In all cases we had level 8 of parallel execution:

  • SDO_CONTAINS(C.GEOMETRY, GET_GEOMETRY(LNG,LAT)) = 'TRUE' returned in 387.7 sec.
  • SDO_ANYINTERACT(GET_GEOMETRY(LNG,LAT),c.geometry)='TRUE' returned in 4.8 sec.
  • SDO_ANYINTERACT(c.geometry,GET_GEOMETRY(LNG,LAT))='TRUE' returned in 389.2 sec.
  • SDO_INSIDE(GET_GEOMETRY(LNG,LAT),c.geometry)='TRUE' returned in 5.0 sec.

This is 80 times faster!

Both SDO_INSIDE and SDO_CONTAINS return the same number (6,973,650) while both SDO_ANYINERACT return slightly higher number (6,973,846).


So we have 2 winners that can use the better parameters order. 

I will focus on this option:

select count(*) 
from accident a, counties c
where SDO_ANYINTERACT(GET_GEOMETRY(LNG,LAT),c.geometry)='TRUE';


I wanted to see how does the parallelism influence this query performance. I rerun it with various combinations using ADW High connection (bellow 8 ECPU explain plan): 



The results are impressive:



What happens if we increase the amount of data? 

I duplicated the original data 21 times, while moving each data points randomly by 10-100 meters each time towards the center of USA.  I didn't want duplicated locations for the test. 

The result was a table of over 146 million locations. I used similar query and tested it with various amounts of ECPU. Here are the results:







Part 2 

For better performance , before execution, it is advisable to:

  • Validate geometry (fix if needed)
  • Create Index

We validated the Counties table

select name,  sdo_geom.validate_geometry_with_context(GEOMETRY, 0.005) status
from  COUNTIES

and received lots of errors of the type:  13367 [Element <1>] [Ring <1>]

Daniel explained that it is probably due to polygon directionality. It was solved by running the sdo_util.rectify_geometry like this:

update counties set geometry=sdo_util.rectify_geometry(geometry,0.005);

Good code for similar problems is in the blog post by Hans Viehmann here


See also the Spatial Data Validation chapter (slides 3-8) in the  Oracle Spatial Best Practices and Tuning Tips by Daniel Geringer here.


Last part was the index creation. While I covered index creation for  PointsLinesPolygons earlier, I learned there is performance benefit in creating CBTREE organized index for streaming point data. See slides 29, 30 here

Our code was:

INSERT INTO USER_SDO_GEOM_METADATA VALUES (
  'ACCIDENTS',  -- table name
  'ADMIN.GET_GEOMETRY(LNG,LAT)',    -- geometry column name
  SDO_DIM_ARRAY(
    SDO_DIM_ELEMENT('X', -180, 180, 0.005),  -- 'X',[min x],[max x],[tolerance]
    SDO_DIM_ELEMENT('Y', -90, 90, 0.005)),    -- 'Y',[min y],[max y],[tolerance]
   4326           -- indicates longitude/latitude coordinates (coordinate system)
 );


CREATE INDEX ACCIDENTS ON ACCIDENTS_2023_03 ( ADMIN.GET_GEOMETRY(LNG,LAT) )
INDEXTYPE IS MDSYS.SPATIAL_INDEX_V2 PARAMETERS('layer_gtype=point cbtree_index=true');

Please note the INDEXTYPE part that is different from the regular: 
INDEXTYPE IS MDSYS.SPATIAL_INDEX_V2;


P.S. 

One more interesting option to test is the SDO_POINTINPOLYGON function that was described Jean Ihm here and Karin Patange here



Thursday, May 23, 2024

OAS 2024 Recommended Patches

Oracle Analytics Server 2024 (OAS 2024), sometimes name OAS 7.6 is available since March. 

Oracle published Critical Patch Update (CPU) Advisor For Oracle Analytics Server and Oracle Business Intelligence - Updated for April 2024 (Doc ID 2832967.2).

 

UPDATE

 In the above described link there is a bundle patch available (currently, at Aug-2024, it's: Patch 36823847 July 2024 - "OAS Stack Patch Bundle 7.6.0.0.240710, or later" + a Java patch). Use the last bundle patch and not the ones described bellow. The remarks at the end of the post are still relevant.





This list is expected to be updated regularly.

Right now there 2 major parts there: OAS Weblogic & OAS FMw Infrastructure. The first OAS 2024 bundle patch is not available yet.

Under Weblogic we see 

  • Database recommendation for July 2021 PSU or later and Native Network Encryption (NNE) recommendation for the DB.
  • Upgrade Java SE (JDK/JRE) (Patch 18143322 & Note 3012587.1, About Updating the JDK Location After Installing Oracle Analytics Server).
  • Patch 36485713 for Weblogic.

 


Under  FMw Infrastructure we see the following list of patches:

  •  Patch 36402397
  • Patch 36348444+Patch 34809489  or  Patch 36074941+Patch 34809489
  • Patch 36349529  or  Patch 35751917
  • Patch 34542329
  • Patch 36316422  or  Patch 32784652



Few extra remarks:

In OAS 2024 there is a new option of Workbook export to Excel (on Tables and Pivots). To make it work you need to follow Installation Guide and install the NodeJS. 

In OAS 2024 you can use the classic data modeler client for RPD or use the new Semantic Model web interface with similar functionality. If the Semantic Model is missing from the Menu after upgrade from previous versions, you have to grant missing permissions that might have been replaced by the import.  See "After Importing A Snapshot into Oracle Analytics Server (OAS), The Semantic Modeler Feature Is No Longer Available (Doc ID 3011914.1)": Use Grant Permission Assignment and grant 'semanticmodeler.author' permission to your administrator role.


Thank You Itsik Hadar for finding the last 2 bullets.