Some systems are experiencing issues

Past Incidents

19th January 2023

Ticket#10193385 | January 19, 2023 | NTB and SFS| IPLC Connectivity Maintenance, scheduled 2 years ago

Event : IPLC Connectivity Maintenance

Reschedule : January 19, 2023 (Thursday) | 6:00 PM to 8:00 PM (PHT)

Activity Details : Just received an update from our internet service provider that this maintenance has been rescheduled from January 13, 2023, to January 19, 2023 due to unforeseen circumstances.

Service Impact : Latency Increase on NCP IPLC Connectivity

Status: Completed successfully

15th January 2023

No incidents reported

14th January 2023

Network and Internet Ticket#10194374 | January 14, 2023 | PNB,JKB,ASB | Brief Description

Event: Network Interruption

Unscheduled: January 14, 2023 (Saturday) | 12:00 PM (PHT) – 4:35 PM (PHT)

Activity Details: We are experiencing network issues that may limit network and internet connectivity at the above-mentioned sites. We are currently conducting an investigation to determine the root cause of the problem

Service Impact: Network Stability and Internet Connectivity of PNB, JKB, and ASB Site

Status: Monitoring stability

Ticket#10193003 | January 14, 2023| Saint Francis Square | Internet Service Provider Node System Software Upgrade, scheduled 2 years ago

• Event: Internet Service Provider Node System Software Upgrade

• Schedule: January 14, 2023 (Saturday) | 12:00 Midnight to 6:00 AM (PHT)

• Activity Details: One of our internet service provider will be upgrading the current software version on one of their Node System to support uplink expansion and addressing possible congestion so it can provide a more stable link and reliable network connection. During the maintenance window, there will be fifty (50) minutes of maximum downtime to the affected internet circuit but due to the way our network is setup, all traffic will be switching over to another available internet circuit uplink and user’s onsite will experience a partial impact on internet and network connectivity.

•Service Impact : Partial Impact on Network and Internet Connectivity at Saint Francis Square Site

• Result: Activity Completed, without issues.

13th January 2023

Ticket#10194267 | January 13, 2023 | All Sites| Intermittent Wireless Authentication Issue

Event : Intermittent Wireless Authentication Issue

Unscheduled : January 13, 2023 (Friday) | 5:00 PM (PHT) – 9:00 PM (PHT)

Activity Details: Our authentication servers are currently having reoccurring which affects the user’s wireless devices having difficulty trying to connect to our internal wireless network connection. We are already doing some further investigation and necessary troubleshooting together with our systems engineering and administration team towards a permanent resolution.

Service Impact: Wireless Devices Connecting to EMAPTA Wireless Network – All Sites

Status: Monitoring stability

12th January 2023

Ticket#10193971 | January 12, 2023 | IPLC | IPLC Connectivity Outage, scheduled 2 years ago

• Event : IPLC Connectivity Outage

• Schedule : January 12, 2023 (Thursday) |1:30 PM (PHT) - 1:52 PM (PHT)

• Details: IPLC is currently down and has been reported to our Internet Service Provider for further investigation.

• Service Impact: Increased latency on VPCs

• Status: Restored

• RFO: Due to ongoing activity changing the network card on their network devices

Ticket#10193694 | January 12, 2023 | All EMAPTA Core and Talents | FortiClient VPN Update and New Password Reset Tool Deployment, scheduled 2 years ago

• Event : Planned Systems Engineering and Administration Deployment - Scheduled

• Primary Schedule : January 12, 2023 (Thursday) | 12:00 Midnight to 6:00 AM (PHT)

• Activity Details : FortiClient VPN Update, Password Reset Tool Deployment, Change System Tray Icon and Redirect Empower to new PWD Change Tool (ECL Laptops Only)

• Service Impact: For all EMAPTA Core and Talents.

• Result: Activity Completed without issues.

11th January 2023

Ticket#10193978 | January 11, 2023| All Sites | Intermittent Wireless Authentication Issue

Event : Intermittent Wireless Authentication Issue

Schedule : January 11, 2023 (Wednesday) | 5:00 PM (PH Time) – Ongoing

Activity Details: Our authentication servers are currently having some issues since yesterday which affects the user’s wireless devices having difficulty trying to connect to our internal wireless network connection. We are already doing some further investigation and necessary troubleshooting together with our systems engineering and administration team towards a resolution.

Service Impact : Wireless Devices Connecting to EMAPTA Wireless Network

Status: Resolved

Fix: Reinstallation of CBOX Servers.

10th January 2023

INFINIVAN Ticket#10193455 | January 10, 2022| All Sites| Partial Network Outage

• Event: Unplanned Internet Service Provider Network Outage.

• Schedule: January 10, 2022 (Tuesday) |8:19 PM (PHT) - 8:25 PM (PHT) • Details: One of our uplinks was observed to be down and is now operational. We have already raised this with our service provider and are waiting for their response. Given that we have another circuit uplink available to our gateway. Just a minimal to NO impact has been anticipated. • Service Impact: Partial to NO Impact on Network and Internet Connectivity

Ticket# 10193416|January 10, 2023| MS Teams Users | Microsoft Teams Degraded Service Functionality

Event : Microsoft Teams Degraded Service Functionality

Unplanned Schedule : January 10, 2023 (Tuesday) | 5:43 PM - Ongoing

Activity Details : Title: Delays in sending and receiving messages and unable to create chats or channels

User Impact: User may have experienced delays in sending and receiving messages and were unable to create chats or channels.

Final Status: We’ve monitored the service and received further confirmation from previously affected users that impact has been remediated.

Scope of Impact: The issue could have affected any user attempting to use the affected features from the Microsoft Teams Desktop, Web and Mobile Clients.

Preliminary Root Cause : Our investigation identified that a backend database latency issue caused timeouts and delays within the service that sends message in Team.

Service Impact: MS TEAMS Users

Event : Microsoft Teams Degraded Service Functionality

Unplanned Schedule : January 10, 2023 (Tuesday) | 5:43 PM - Ongoing

Activity Details : Please see below details of Microsoft announcement regarding MS Teams functionalities : Title: Users may be unable to send and receive messages in Microsoft Teams Chats or Channels User Impact: Users may be unable to send and receive messages in Microsoft Teams Chats or Channels Current Status: We're reviewing service monitoring telemetry to isolate the root cause and develop a remediation plan. Scope of Impact: Impact is specific to users who are served through the affected infrastructure.

Service Impact: MS TEAMS Users

9th January 2023

No incidents reported

8th January 2023

No incidents reported

7th January 2023

No incidents reported

6th January 2023

Ticket#10192559 | January 6, 2023 | GDB | Network Connectivity Outage

• Event: Unplanned Network Connectivity Outage

• Schedule: January 6, 2023 (Friday) | 8:07PM (PHT) - 10:02 PM (PHT)

• Details: Already reported to Service Provider for further investigation.

• Service Impact: Site Internet Connectivity Downtime.

• Status: Restored

• Root Cause: TBA

Ticket#10192490 | January 6, 2023 | IPLC | Unplanned IPLC Outage

• Event : IPLC Connectivity Outage

• Schedule : January 6, 2023 (Monday) |11:44 AM (PHT) - 12:56 PM (PHT)

• Details: IPLC is currently down and has been reported to our Internet Service Provider for further investigation.

• Service Impact: Increased latency on VPCs

• Status: Restored

• Root Cause: Outage on Philippine Backhaul

Ticket#10192475 | January 06, 2022 | All Sites| Wireless Authentication Issues

• Event : Intermittent Wireless Authentication Issue

• Schedule :Unplanned | January 06, 2023 (Friday) | 8:25 AM – 11:05 AM (PH Time) - Resolved

• Activity Details :Our systems engineering and administration (SEA) team has already resolved the issue by restoring the certificate files from the backup folder, re-applying TLS settings, and restarting the server service. Our authentication servers have been monitored up and stable for about 3 hours now. Incident report will be provided upon request.

Result: Resolved

• Event : Intermittent Wireless Authentication Issue

• Schedule :Unplanned | January 06, 2023 (Friday) | 8:25 AM – Ongoing (PH Time)

• Activity Details : Our authentication servers are currently having some issues which affects the user’s wireless devices having difficulty trying to connect to our internal wireless network connection. We are already doing some further investigation and necessary troubleshooting together with our systems engineering and administration team towards a resolution.

Service Impact : Wireless Devices Connecting to EMAPTA Wireless Network – All Sites