Place shrimp, celery, pickling spices, and salt in a large saucepan. Cover with boiling water. Cover and simmer just until shrimp turns pink, 3 to 5 minutes. Drain and discard cooking liquid and seasonings.
Run shrimp under cold water, peel and devein, then in a large, shallow, nonreactive bowl alternate shrimp with onion. Place bay leaves on top. Combine oils, vinegar, capers, celery seed, paprika, mustard, and cayenne and pour over shrimp and onions. Cover and place in refrigerator for 24 hours, turning twice during marinating time. Remove shrimp and onions from marinade. Place on serving platter and garnish with fresh parsley. Serve with toothpicks.