Dealing with Docker Memory Issues

In this article, I just want to discuss two of the major memory issues we have faced while working with dockers.


1. Default docker uses the base directory as /var/lib/docker and due to memory constraints many times we need to change this default to some other path. There are two ways to do this- One is to go with the "symlink" to refer some other directory and another way is to use docker provided "-g" option. The problem with the first approach is that after every docker service restart, the symlink directory is getting unmounted and to mount again we have to restart the host machine.


To resolve this, using second approach, we have to use '-g' option that docker provides. In Centos the guide says we should edit /etc/sysconfig/docker, and add the -g option in the other_args variable: ex. other_args="-g /var/lib/testdir" but this was not getting picked somehow in our AWS server. So after some research we have included this option in the UCD start docker component itself.

2. Second issue related to memory is that by default for any docker container the root is assigned 10gb of space and for WAS installation more than 10GB space is required. For this we need to append another attribute --storage-opt dm.basesize=25G. This will fix the problem.

To fix above, our final docker start command looks like
nohup sudo /usr/bin/docker daemon -g /opt/NewDirectory --storage-driver=devicemapper --storage-opt dm.basesize=25GB --storage-opt dm.loopmetadatasize=4GB --insecure-registry HN_OR_IP_ADDR_HERE:5000 --log-level="debug" >dockerlog.txt

Some good links on the above:

• Change docker base directory
• Change default container memory

Invoking PayPal Classic API's- Signature and Certificate based approahes

 Paypal provides two ways to invoke their API's- Through the signature base and another is through certificates.

1. Signature based: For signature based the mandatory parameters are:
    - API Username
    - API password
    - API signature

2. Certificate based: Mandatory parameters for this approach are:
    - API Username
    - API password
    - API certificate
    - API certificate key


Generate API Signature and Certificate

You need to generate the signature or certificate depending upon the approach you are going to use. The process for both is nearly same. Login to your Paypal sandbox account. If you don't have the valid account, register as new user. For more details for generating certificates and signatures can be obtained from here.

Code snippet to be integrated

1. Define property file

    - Property file for Signature based approach

        acct1.UserName = ****-facilitator_api1.gmail.com
        acct1.Password = 348889***
        acct1.Signature = W19Jweloyt7bsmlaqzxrL7vYoFV*****

    - Property file for Certificate based approach

        acct1.UserName = ****-facilitator_api1.gmail.com
        acct1.Password = 348889***
        acct1.CertPath=/opt/tmp/paypal_cert.p12
        acct1.CertKey=keyvalue

    - Common properties for both the approaches
        
        service.EndPoint.PayPalAPI=https://api.sandbox.paypal.com/2.0/
        service.EndPoint.PayPalAPIAA=https://api.sandbox.paypal.com/2.0/
        service.EndPoint.Permissions=https://svcs.sandbox.paypal.com/
        http.ConnectionTimeOut=5000

2. Initialize properties: here we are taking example of the certificate based approach

    loadProperties(){
    Properties PAYPAL_PROP = new Properties();
    PAYPAL_PROP.put("acct1.UserName",readProperty("acct1.UserName"));
    PAYPAL_PROP.put("acct1.Password",readProperty("acct1.Password"));
    PAYPAL_PROP.put("acct1.CertKey", readProperty("acct1.CertKey"));
    PAYPAL_PROP.put("acct1.CertPath", readProperty("acct1.CertPath"));
    PAYPAL_PROP.put("service.EndPoint.PayPalAPI", readProperty("service.EndPoint.PayPalAPI"));
    PAYPAL_PROP.put("service.EndPoint.PayPalAPIAA", readProperty("service.EndPoint.PayPalAPI"));
    PAYPAL_PROP.put("service.EndPoint.Permissions", readProperty("service.EndPoint.Permissions"));
    PAYPAL_PROP.put("http.ConnectionTimeOut", .readProperty("http.ConnectionTimeOut"));
    }


3. Invoke API's: Sample code snippet in Java for doVoid request

    DoVoid(String sAuthorizationId){
        DoVoidReq doVoidReq = new DoVoidReq();
        DoVoidRequestType doVoidRequest = new DoVoidRequestType(sAuthorizationId);
        doVoidReq.setDoVoidRequest(doVoidRequest);
        PayPalAPIInterfaceServiceService service = null;
        service = new PayPalAPIInterfaceServiceService(PAYPAL_PROP);
        DoVoidResponseType doVoidResponse = service.doVoid(doVoidReq);
}


Similarly you can invoke other Paypal API's using the above code.

SHAPE—an approach for self-healing and self-protection in complex distributed networks

Abstract

Increasing complexity of large scale distributed systems is creating problem in managing faults and security attacks because of the manual style adopted for management. This paper proposes a novel approach called SHAPE to self-heal and self-protect the system from various kinds of faults and security attacks. It deals with hardware, software, and network faults and provides security against DDoS, R2L, U2L, and probing attacks. SHAPE is implemented and evaluated against various standard metrics. The results are provided to support the approach.

You can access detail paper from here.

Keystore based approach for Key Management through Java

Java keystores provide a convenient mechanism for storing and deploying public and private keys. Truststore and Keystore file will be used in the communication to provide secured transaction between two systems. The keytool command is used to create the key store file which contains the public/private keys and then using keystore, Create a truststore file which contains only public keys. Keystore will keep the private key secure by adding the additional two level password securities at both the keystore and at private key level.

Here we will be using the RSA based asymmetric approach for encryption and decryption.

Steps to follow:

1. Generate a private key in keystore file and verify the newly created keystore file.  While generating the keystore and key, we need to define the different passwords.

keytool -genkeypair -alias certificatekey -keyalg RSA -validity 3650 -keystore keystore.jks

We can define the validity also for which this key will remain valid. This we can use for keeping the check for keys to remain valid for a year or so.

keytool -list -v -keystore keystore.jks

2. Export the certificate (describing public key) and import the certificate into the truststore file. Verify the newly created trust store file.

keytool -export -alias certificatekey -keystore keystore.jks -rfc -file selfsignedcert.cer

keytool -import -alias certificatekey -file selfsignedcert.cer -keystore truststore.jks

3. Keystore will remain at OMS for decryption and Truststore will be shared to DAX for encryption.
4. Java custom code will fetch the private key details from the keystore. Sample program for encryption and decryption will look like:

Note: 

1. Share only the truststore.jks i.e. public key with other system. Keystore.jks can perform both encryption and decryption and should never be shared.

2. Don't commit the private key into code repository. It should be placed at secure place so as to maintain complete security.

/* getting data for keystores */

//for decryption

File file = new File("{Path}\\keystore.jks");

//for encryption

File file1 = new File("{Path}\\truststore.jks");

FileInputStream is = new FileInputStream(file);

KeyStore keystore = KeyStore.getInstance(KeyStore.getDefaultType());

FileInputStream is1 = new FileInputStream(file1);

KeyStore keystore1 = KeyStore.getInstance(KeyStore.getDefaultType());

/* Information for certificate to be generated */

String password = "{Password given at time of key generation}";

String alias = "certificatekey";

           

/* getting the private key */

keystore.load(is, password.toCharArray());

PrivateKey key = (PrivateKey) keystore.getKey(alias,password.toCharArray());

           

/* Get certificate of public key */

keystore1.load(is1, password.toCharArray());

java.security.cert.Certificate cert = keystore1.getCertificate(alias);

/* Here it prints the public key */

System.out.println("Public Key:");

System.out.println(cert.getPublicKey());

/* Here it prints the private key */

System.out.println("\nPrivate Key:");

System.out.println(key);

Cipher cipher = Cipher.getInstance("RSA");

cipher.init(Cipher.ENCRYPT_MODE, cert.getPublicKey());

byte[] encryptedData = cipher.doFinal("Text to be Encrypted".getBytes());

System.out.println("Encryted Data: " + encryptedData);

           

           

cipher.init(Cipher.DECRYPT_MODE, key);

byte[] descryptedData = cipher.doFinal(encryptedData);

System.out.println("Decrypted Data: " + new String(descryptedData));

1

Reactivate Oracle Expired Passwords ORA-28001

This is because password have reached 180 Default limit for Password life time (ORA-28001).

To reset user:

1. Connect to database using any admin users.

2. Execute 
Sql > select * from dba_profiles;
PASSWORD_LIFE_TIME field is responsible for expiring of password after 180 days.

3. Execute following command to disable this feature:
Sql> ALTER PROFILE DEFAULT LIMIT PASSWORD_LIFE_TIME UNLIMITED;

4. Now crosscheck for disabling of this feature.
Sql > select * from dba_profiles;
The value in PASSWORD_LIFE_TIME has changed to unlimited. Now password will never expire.

5. Now change the password of locked user and unlock using following.
sql> alter user [user_name] identified by [password];

sql> alter user [User_name] account unlock;

6. Crosscheck by value of accout_status field in dba_users view.
sql> select username,account_status from dba_users;
The value of account_status filed should by "OPEN" for corresponding user.

Kindle Fire Application Development Limitations

In an IDC survey of more than 2,000 developers worldwide, the Amazon Kindle Fire garnered the second-most interest from developers among 15 Android tablets — second only to the Samsung Galaxy Tab. In North America, it was the top tablet, and the percentage of developers who said they were interested in developing for it was just four points less than the percentage who were interested in developing for the iPad before its launch in April 2010.

The tablet already lacks features such as a camera, GPS and an external microphone that render many apps useless. There are “No Google” services for Kindle. The modified Android for Kindle is different from the actual Android OS. Based upon the Android 2.3.4 OS, Kindle OS lacks following:

• Google services like C2DM (for push notification), Google Maps, and Location based API’s are not available.
• Amazon has its own Amazon store for the applications. No Android Store.
• Debugging the Kindle app from developers perspective requires little tweaking.
• No Android updates, like upgrading to HoneyComb or Ice Cream Sandwich.
• Limited Hardware support as Camera, GPS and Microphone are not available.
• Changed User Interface from the default Android provides UI.

For developers who want to use the Push Notification or GPS/Location based features, have to use some external tools to program their application.

• Push notifications: For using push notifications, one can use Urban Airship helium. Helium is Urban Airship’s proprietary push messaging service, and is the default transport for the library.

           URL: http://urbanairship.com/docs/android-client-push.html

• Location: Amazon Fire supports Skyhook sdk, which can be used for determining the location. Skyhook collect raw data from Wi-Fi access points, GPS satellites and cell towers with advanced hybrid positioning algorithms. By leveraging the strengths of more than one underlying position technology, Skyhook's Core Engine provides the best possible location available in any environment.

           URL: http://www.skyhookwireless.com/

• Maps: MapQuest SDK for Android can be used. MapQuest has been providing valuable solutions to solve business challenges since 1997, with a focus on reliability and service. They were the first major mapping site on the internet.

           URL: http://developer.mapquest.com/web/products/beta/android

Debug Kindle Fire Application

For Windows users

Follow the above instruction and the adb_usb.ini file is located here:

1. Use a text editor and open the file adb_usb.ini. It's located at c:\Users\[User-Name] \.android\adb_usb.ini.

2. Then add 0x1949 to the end of the file.

3. Next save the file.

4. Next you will need to edit the file android_winusb.inf file. Use a text editor and open up this file 

C:\Program File(x86)\Android\andorid-sdk\extras\google\usb_driver 

(note: if you install your android sdk somewhere else then go to that location). When you installed Android SDK, the USB Driver package will also be there, if not you will have to download it. Note: if you can't edit the file ensure you have write permission to do so.

5.  Under section [Google.NTx86] and [Google.NTamd64] add the following:

;Kindle Fire
%SingleAdbInterface% = USB_Install, USB\VID_1949&PID_0006
%CompositeAdbInterface% = USB_Install, USB\VID_1949&PID_0006&MI_01

6. Now save the and close the file.

When you connect the Kindle Fire to the PC, it will not recognize the device hardware so you will need to install the USB driver. Here's how:

1. Using the USB cable plugin the Kindle Fire to the computer.

2. Open Device Manager from My Computer.

3. Open Other devices on the right pane.

4. Right click Kindle > Update Driver Software.

5. The click on Browse my computer from driver software.

6. Next click on Browse button, and find the file android_winusb.inf at C:\Program Files 
(x86)\Android\andorid-sdk\extras\google\usb_driver. If you install the SDK somewhere else then go to that location. If you come across any driver errors, then download the drivers for Win7 from here. Unzip and point the setup to this directory home.

7. Now click Next.

8. Select Install this driver software anyways. 

9. Now check your Device Manager and see if you could see Android Phones > Android Composite ADB Interface.  

10. Now you can restart the ADB Server to get an update.

11. Go to the folder android-sdk\platform-tools\.

12. Run adb kill-server.

13. Run adb start-server.

14. Run adb devices.

For MAC User

1. From the Home screen tap on the Quick Settings  icon.

2. Next select More.

3. Select Device.

4. Switch on "Allow Installation of Application From Unknown Sources".

5. Next connect the Kindle Fire to the MAC.

6. Then use an editor to open the file ~/.android/adb_usb.ini.

7. Now you will need to add the Vendor ID text "0x1949" at the end of the file.

8. Save the adb_usb.ini file.

9. Then restart the adb server process with this command "adb kill-server" and "abd start-server". This will restart the process and read the new value from the file.

10. Now run "adb devices" to see your device appears.

You may need to have your computer restart to see the changes.